[HELP] Hybrid introducing audio delay that is not present in source material

[4i4oVi40]

40ms is one frame with 25fps which some folks notice especially if they are singers and look at themself singing in a video.
Happy the explaination helped to understand what's going on and yes, it's a feature (which like mentioned before can be disabled) not a bug. 

Cu Selur

In "The mystery of delayed AAC audio tracks" I read that although "the FDK-AAC encoder adds an offset of 1024 samples per channel (like 42,66 ms for stereo at 48 kHz), the player syncs all streams based on their timestamps and everything should be perfectly in sync. The offsets are ignored."
https://www.voukoder.org/forum/thread/56...io-tracks/
(I read similar arguments elsewhere too.)

If this is the case, it seems Hybrid is introducing an unnecessary offset, which the player takes care of anyway. Why should we explicitly compensate for the delay by moving the audio 43 ms ahead, when it seems the VLC player is doing this correction itself?

[Selur]

As far as I know that argument is only correct if you keep the timing of the source, which does not work with Hybrid since it demuxes audio&video the the time stamps in the output container are created by the muxer.
So I don't think Hybrid is doing anything wrong.
-> If you think otherwise simply disable the option.

Cu Selur

[4i4oVi40]

As far as I know that argument is only correct if you keep the timing of the source, which does not work with Hybrid since it demuxes audio&video the the time stamps in the output container are created by the muxer.
So I don't think Hybrid is doing anything wrong.
-> If you think otherwise simply disable the option.

Cu Selur

I created a short (6 sec) test avi file (24 fps) where the sound (PCM) starts exactly at the 2nd sec.
I encoded it (x264) directly with ffmpeg using: -c:a aac -b:a 192k
Then I encoded it with Hybrid again with AAC audio using 192k.

As usual Hybrid introduced the -43 ms delay:
iId_2_aid_1_DELAY_-43ms_2021-10-13@01_30_17_6910_01.wav

However, the offsets of the audio in both mp4 files are different.
Also, MediaInfo and ffprobe give different info.
The ffmpeg file: Duration: 6 s 0 ms, Source duration: 6 s 21 ms, Source_Duration_LastFrame: -16 ms
The Hybrid file: Duration: 6 s 37 ms (!)

When imported back in the NLE, the ffmpeg audio seems 1 frame earlier, while the Hybrid audio seems 0.5 frame earlier. (See screenshot.)

If the offset formula used in Hybrid is correct, shouldn’t it produce the same audio offset as ffmpeg?

Here’s the MediaInfo for the audio streams of the 2 files:


ffmpeg Audio

ID                                      : 2
Format                                  : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                : mp4a-40-2
Duration                                : 6 s 0 ms
Source duration                          : 6 s 21 ms
Source_Duration_LastFrame                : -16 ms
Bit rate mode                            : Variable
Bit rate                                : 66.2 kb/s
Maximum bit rate                        : 192 kb/s / 192 kb/s
Channel(s)                              : 2 channels
Channel layout                          : L R
Sampling rate                            : 48.0 kHz
Frame rate                              : 46.875 FPS (1024 SPF)
Compression mode                        : Lossy
Stream size                              : 48.6 KiB (9%)
Source stream size                      : 48.6 KiB (9%)
Default                                  : Yes
Alternate group                          : 1
mdhd_Duration                            : 6000

Hybrid Audio

ID                                      : 2
Format                                  : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                : mp4a-40-2
Duration                                : 6 s 37 ms
Bit rate mode                            : Variable
Bit rate                                : 66.2 kb/s
Maximum bit rate                        : 192 kb/s
Channel(s)                              : 2 channels
Channel layout                          : L R
Sampling rate                            : 48.0 kHz
Frame rate                              : 46.875 FPS (1024 SPF)
Compression mode                        : Lossy
Stream size                              : 48.6 KiB (16%)
Encoded date                            : UTC 2021-10-12 22:30:36
Tagged date                              : UTC 2021-10-12 22:30:36

[Selur]

If the offset formula used in Hybrid is correct, shouldn’t it produce the same audio offset as ffmpeg?

Why should that be the case?
a. Hybrid by default does not use FFmpeg for muxing (you can tell it to do so by enabling 'Config->Containers->Mp4 Settings->FFmpeg instead of MP4Box').
b. Just looking at the screenshot 0.5 frame (Hybrid) vs. 1 frame (FFmpeg), Hybrids way seems to be more correct. Also looking at your screenshot it seems like FFmpegs delay is even more than one frame.

To be clear, Hybrid never aimed to behave like FFmpeg, I wrote that 'encoder compensation delay adjustment'-code to ideally totally compensate the encoder delay. I never looked at what ffmpeg or any other tool where I could look at the source code did since I only followed the format and encoder infos I could gather.

While at it you could also:

• post what MediaInfo reports about your souce avi.
  
• check what happens if you:
  • use FFmpeg in Hybrid as muxer
    
  • disable the delay handling in Hybrid while using MP4Box and FFmpeg as muxer
    

Also please:
a. use 'code'-tags or 'quote'-tags when posting MediaInfo outputs or similar, it makes the whole thing more readable. (I added 'code'-tags to your last post)
b. use the detailed MediaInfo output of the whole file. (Container and a/v delay infos are interessting and important)

The ffmpeg file: Duration: 6 s 0 ms, Source duration: 6 s 21 ms, Source_Duration_LastFrame: -16 ms
The Hybrid file: Duration: 6 s 37 ms (!)

Duration = Container stream length
Source Duration = Stream length
Source Duration LastFrame = no clue how that should be interpreted (you would have to ask the MediaInfo author about that)
That does not really match the screenshot, so another question is whether:
a. the results reported by MediaInfo are correct
b. the handling of your NLE regarding the input (mp4 and avi) are correct (are those delays properly respected?)
-> the whole MediaiInfo and FFProbe info would be needed.

Cu Selur

[4i4oVi40]

• check what happens if you:
  • use FFmpeg in Hybrid as muxer
    
  • disable the delay handling in Hybrid while using MP4Box and FFmpeg as muxer
    

I select FFmpeg as muxer at:
Config > Containers > FFmpeg instead of MP4Box

However how do I disable the delay? I tried:
Config > Input > Ignore delay from input
But it doesn't make any difference. Both MP4Box and FFmpeg don't seem to be affected by "Ignore delay from input".

[Selur]

However how do I disable the delay? I tried:
Config > Input > Ignore delay from input

'Ignore Delay from Input' only does that it ignores any input delay. If your source already has a delay of 0 it naturally doesn't change anything.

You can tell Hybrid to not compensate the aac encoder delay by deactivating "Audio->Misc->Compensate aac encoder delay" (haven't disabled this for years, so if there's a problem let me know) before the entries in the audio job queue are created.

source: second post in this thread,...

Cu Selur

[4i4oVi40]

You can tell Hybrid to not compensate the aac encoder delay by deactivating "Audio->Misc->Compensate aac encoder delay" (haven't disabled this for years, so if there's a problem let me know) before the entries in the audio job queue are created.

source: second post in this thread,...

Cu Selur

Thanks for the tip! And sorry I didn't see it in the beginning of the thread.

I did all the tests you suggested. The Hybrid's steps are:
1. extracting audio as pcm/wav using ffmpeg
2. encoding audio to aac using fdkaac
3. encoding video to MPEG-4 HEVC with x265
4. (mkv) creating mkv/webm file with mkvmerge
4. (mp4:MP4Box) creating mp4 file with MP4Box
4. (mp4:ffmpeg) creating mp4 file with ffmpeg

There are 3 ways to create the container and 2 audio delay options (compensate or not). So 6 cases total.
Only in one of them (mp4:MP4Box with delay compensation), an additional step is introduced before creating the container:
renaming iId_2_aid_1_DELAY_-41ms.aac to iId_2_aid_1_ms.aac
This seems to REVERSE the delay compensation.

My question is: Why is this done? And why only for the mp4 output but not for the mkv output?

In short, the results of the tests are as follows:

Compansate aac encoder delay:
+ MP4Box_delay-41ms.mp4  //appears as the source in VirtualDub2
mkvmerge_delay-41ms.mkv  //audio 1.5 frame ahead
ffmpeg_delay-41ms.mp4  //audio 1 frame ahead

Do NOT compansate aac encoder delay:
+ MP4Box_delay0ms.mp4  //appears as the source in VirtualDub2
+ mkvmerge_delay0ms.mkv  //appears as the source in VirtualDub2
ffmpeg_delay0ms.mp4  //video 2 frames ahead and audio 1 frame ahead

My personal conclusions for muxing AAC:
1. For mp4, stick with defaults (MP4Box, Compansate aac encoder delay).
2. For mkv, uncheck "Compansate aac encoder delay".
3. Do NOT bother with "FFmpeg instead of MP4Box.

Later I may post the detailed MediaInfo data for all the 6 tests and screenshots of VirtualDub2, visualizing the Audio vs. Video delays.

[Selur]

Only in one of them (mp4:MP4Box with delay compensation), an additional step is introduced before creating the container:
renaming iId_2_aid_1_DELAY_-41ms.aac to iId_2_aid_1_ms.aac

No clue without a debug output. I probably could search which code paths might be the cause of this, but searching blindly will take me hours,..
--

What's confusing is that if mkv really removed the delay before muxing and doesn't use it, shouldn't the output be the same as when the delay isn't used?
-> like I wrote, need a debug output to look into the code.


Cu Selur

Ps.: might also be interessting whether the observersions are still true when he-aac and he-aacv2 are used instead of lc-aac (<- I assume you used lc-aac, since you didn't mention what ou are using,...)

[4i4oVi40]

Only in one of them (mp4:MP4Box with delay compensation), an additional step is introduced before creating the container:
renaming iId_2_aid_1_DELAY_-41ms.aac to iId_2_aid_1_ms.aac

No clue without a debug output. I probably could search which code paths might be the cause of this, but searching blindly will take me hours,..
--

What's confusing is that if mkv really removed the delay before muxing and doesn't use it, shouldn't the output be the same as when the delay isn't used?
-> like I wrote, need a debug output to look into the code.


Cu Selur

Ps.: might also be interessting whether the observersions are still true when he-aac and he-aacv2 are used instead of lc-aac (<- I assume you used lc-aac, since you didn't mention what ou are using,...)

Here's the DebugOut file. And the renaming is at

Line 961: newRenameJob called for D:\Temp\iId_1_aid_1_DELAY_-41ms_2021-10-15@02_00_15_2510_02.aac to D:\Temp\iId_1_aid_1_ms_2021-10-15@02_00_15_2510_02.aac

[Selur]

thanks will look at it after work and report back