[HELP] Hybrid introducing audio delay that is not present in source material

[4i4oVi40]

I looked at the DebugOut file and then did some more testing.

Now it seems to me that the "Compensate aac encoder delay" option does nothing when creating a mp4 file with MP4Box. Whether or not it's checked, Hybrid produces exactly the same mp4 file.

I took the CLI options and tried to recreate the process outside of Hybrid like that:

ffmpeg -i foo.avi -map 0:1 -vn -sn -ac 2 -ar 48000 -acodec pcm_s16le -f wav -map_metadata -1 - | fdkaac --bitrate-mode 4 --profile 2 --transport-format 0 --gapless-mode 2 --ignorelength -o foo.aac -

ffmpeg -noautorotate -nostdin -i foo.avi -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -f yuv4mpegpipe - | x265 --output-depth 10 --y4m --profile main10 --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --lookahead-slices 0 --crf 28.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --output foo.265 -

Then I muxed with MP4Box, with and without delay.

MP4Box -add foo.265:name="" -brand hvc1 -add foo.aac -mpeg4 -new foo.mp4

MP4Box -add foo.265#video:fps=24:delay=41:name="" -brand hvc1 -add foo.aac -mpeg4 -new foo_.mp4

Again, the resulting mp4 files are exactly the same.

Also, mkvmerge produces mkv with synced video and audio only when "Compensate aac encoder delay" is not checked.  So I start to think that keeping "Compensate aac encoder delay" unchecked for both mp4 and mkv is actually preferable.

[Selur]

That seems strange.

MP4Box -add foo.265#video:fps=24:delay=41:name="" -brand hvc1 -add foo.aac -mpeg4 -new foo_.mp4

should delay the video by ~ 1frame.

-> I will do some testing.

Cu Selur

[Selur]

using a delay of 1000ms (- encoder compensation) I get:

MP4Box -add "E:\Temp\2021-10-15@13_30_55_0410_04.265"#video:fps=25:name="":lang="" -brand hvc1 -add "E:\Temp\iId_6_aid_1_DELAY_950ms_2021-10-15@13_30_55_0410_02.aac"#audio:delay=950:name="":lang="" -mpeg4 -itags tool="Hybrid 2021.10.10.1" -tmp "E:\Temp" -new "E:\Output\2021-10-15@13_30_55_0410__05.mp4"

and the audio plays delayed by ~1s.

Using a delay of -1000ms (-encoder compensation) I get:

MP4Box -add "E:\Temp\2021-10-15@13_33_12_2410_04.265"#video:fps=25:delay=1050:name="":lang="" -brand hvc1 -add "E:\Temp\iId_7_aid_1_ms_2021-10-15@13_33_12_2410_02.aac"#audio:name="":lang="" -mpeg4 -itags tool="Hybrid 2021.10.10.1" -tmp "E:\Temp" -new "E:\Output\2021-10-15@13_33_12_2410__06.mp4"

and audio plays ~1s too early.

So unlike your example, here the delay seems to work. (I checked I'm using the same MP4Box&Co version that I packed with the dev version.)
How do attached files look to you? Are they delayed or do they play (and look) in sync?

Cu Selur

[Selur]

Since I'm using a newer version of MP4Box than with the last public Hybrid release (they fixed a problem with the handling or frame rate), I send you a link to my current dev version.
I suspect that the frame rate issues migh also caused audio delay problems.
(I also attached the MP4Box version in case other Windows users also run into this.)

Cu Selur

[Selur]

I did a quick test with 5.1 audio, MP4Box and muxer:
(assuming VirtualDub2 properly loads and displays the audio; if this is wrong everything that follows is false)
Original:

encoded with FDK:

Encoded with FAAC:

Encoded with FHG

Encoded with Nero:

Encoded with QAAC:


So to me it looks like:
Original vs FAAC_no_encoder_compensation; audio moved half a frame backward
Original vs FDK_no_encoder_compensation: no delay
Original vs. FHG_no_encoder_compensation: audio moved ~3/4 frame backwards
Original vs Nero_no_encoder_compensation: audio moved one and a half frame backwards
Original vs QAAC_no_encoder_compensation: audio moved one and a half frame backwards

Would you agree with these findings based on the images?
Can you reproduce these findings (I attached the source)?

Assuming the findings are correct, then:
a. the only really good delay handling happens with FDK_no_encoder_compensation.
b. yes, in encoder compensation is needed since using no_encoder_compensation isn't correct.

---
General problem:
a. no clue how accurate VirtualDub2 is (for exact adjustments a single ms accurate audio wave form would be needed)
b. to adjust the encoder delay a ton of extensive testing needs to be done so that the delay can be adjusted for each decoder depending on sampling rate, encoding mode and may be channel count. + muxer (FFmpeg/MP4Box and container mp4/mkv/...) + whether or not the input already has a delay.

Cu Selur

Ps.: Note that a delay of ~1 frame is usually not visible during casual playback this is only reall interessting if you encode to aac and plan to use the output as a editing base for later projects.
Even experts have can't usually see (or more like feel) this unless they are really familiar with the source. Singers sometimes spot the problem when they see frame by frame clips with underlayed audio waves.
So this isn't a really concerning matter.

[4i4oVi40]

So unlike your example, here the delay seems to work. (I checked I'm using the same MP4Box&Co version that I packed with the dev version.)
How do attached files look to you? Are they delayed or do they play (and look) in sync?

Yes, you're absolutely right. Your files indeed play (in VLC) and look (in VirtualDub2) delayed as they should be.

Nevertheless, I ran one more test with 1000 ms delay like that:

MP4Box -add foo.265#video:fps=24:delay=1000:name="" -brand hvc1 -add foo.aac -mpeg4 -new foo1000.mp4

And once again the resulting file comes with NO delay with video and audio perfectly in sync (audio starts on the 2 sec. like in the original file).

I used MP4Box from your package Hybrid, build: GPAC version 1.1.0-DEV-rev1089-gb4d25ac3b-ab-suite

Here I'm attaching my video and audio files (foo.265 and foo.aac) and the resulting file. Also, a screenshot.

Can you please try my files with the MP4Box muxing command above, and let me know what I'm doing wrong. This is really weird...

P.S. I just saw your last 2 posts. I downloaded the new MP4Box and your test files and I'll do the tests that you suggested. And, yes, I totally agree with everything you wrote.

[Selur]

Can you please try my files

What files? Did I miss something? I haven't seen a file from you,...
(You need to zip them and attach them)

[4i4oVi40]

Since I'm using a newer version of MP4Box than with the last public Hybrid release (they fixed a problem with the handling or frame rate), I send you a link to my current dev version.
I suspect that the frame rate issues migh also caused audio delay problems.
(I also attached the MP4Box version in case other Windows users also run into this.)

YES, it was MP4Box! I replaced it and now the delay comes where it should be. See screenshot.

So it wasn't Hybrid or VirtualDub2, it was MP4Box that was buggy. You should probably replace it in the downloadable Hybrid release too...

So now when the culprit is identified and replaced, I can continue with the tests you suggested to replicate your results.

---

Can you please try my files

What files? Did I miss something? I haven't seen a file from you,...
(You need to zip them and attach them)

Sorry, here they are... Although, it's a moot issue now when it was proven that MP4Box was causing the problem.

[Selur]

I think there is still something 'off' with that MP4Box version.

I got a video where VirtualDub2 shows this wavefront: https://ibb.co/v4YpQyv

I then reencoded the audio with faac and used mp4box (MP4Box - GPAC version 1.1.0-DEV-rev1314-gb9b5abb0e-ab-suite) for muxing.

First with no additional delay:

MP4Box -add "E:\Output\video.m4v"#video:fps=25:name="":lang="" -add "E:\Output\audio.aac"#audio:name="":lang="" -mpeg4 -itags tool="Hybrid 2021.10.15.1" -tmp "E:\Output" -new "E:\Output\faac_nodelay.mp4

this gave me:
https://ibb.co/hKPDVyL
so I need a delay of around half a frame.

Using a delay of 20:

MP4Box -add "E:\Output\video.m4v"#video:fps=25_delay=20:name="":lang="" -add "E:\Output\audio.aac"#audio:name="":lang="" -mpeg4 -itags tool="Hybrid 2021.10.15.1" -tmp "E:\Output" -new "E:\Output\faac_20.mp4

gave me:
https://ibb.co/2vb7Z4S
No effect at all,...

Using a delay of 40:

MP4Box -add "E:\Output\video.m4v"#video:fps=25_delay=40:name="":lang="" -add "E:\Output\audio.aac"#audio:name="":lang="" -mpeg4 -itags tool="Hybrid 2021.10.15.1" -tmp "E:\Output" -new "E:\Output\faac_40.mp4

gave me:
https://ibb.co/wwNZ3rJ
No effect at all,...

-> I'll compile a new MP4Box version based on the latest code over night and will do some more testing and if the issue isn't fixed I'll report it to the mp4box authors,..

Cu Selur

[4i4oVi40]

Using a delay of 40:

MP4Box -add "E:\Output\video.m4v"#video:fps=25_delay=40:name="":lang="" -add "E:\Output\audio.aac"#audio:name="":lang="" -mpeg4 -itags tool="Hybrid 2021.10.15.1" -tmp "E:\Output" -new "E:\Output\faac_40.mp4

You have a typo, "_" instead of ":". You have fps=25_delay=40. It should be fps=25:delay=40. This may be the problem.