hierarchical B frames or B-pyramid

[BudgetGamer123]

Hello,
Last year or so I was trying to learn about x264 encoding when I got my new chip, 16thread ryzen 7 3700x.

After a lot of tinkering and reading, source codes forum posts and all, I found something called "b pyramid" and in this website which is no longer available,there was a blog post about it at:
https://www.ramugedia.com/hierarchical-b...-b-pyramid
But to make use of the B Pyramid structure of x264 encoder, GOP length should be multiplies of 2^n like 64 or 128
This trick back then, improved the quality of CBR 6000 stream which I was aiming. Only oddity was using 64 Fps with GOP length of 128

My commandline options for x264 encoder in obs were:

preset=medium x264-params=vbv-bufsize=12000:vbv-maxrate=6000:threads=12:lookahead-threads=1:b-pyramid=strict:bframes=2:opencl=true:sync-lookahead=64:rc-lookahead=32:psy-rd=0.4:aq-strength=0.6:deblock=2,1:trellis=2:filler=1:crf=17

Skipping to present, I was finally able to upgrade the gpu to 3080ti (I upgrade only in 10 years) and now I would like to do the same trick with NVencoder, but FFMPEG doesn'T have too many options, I have tried reading nvencoder header file to get the variables in it but It is not accessible as x264. Unfortunately. Classic nvidia I guess. Most of the stuff I have been trying to add to obs command line option keeps refusing it, and up-to-date variable sheet is nowhere to be found.

Is there a way to tweak gpu encoder to do the same trick? x264 was beautiful with this. I wonder
Thanks
Budget Gamer

[Selur]

Okay, a few things,...

• Sure, the gop size limits the number of potential gain from b-pyramids, but:
  • you usually don't encode with a fixed gop size (you either let the encoder decide to use gop sizes up to a limit, or you let it decide for itself without a limit)
    
  • A b-pyramid of level 1 requires 2-bframes, level 2 requires 3-4 b-frames and level 3 7+. So by liming the number of b-frames you also limit theoretical max level of the pyramid. It rarely makes sense to use too many b-frames or too high of a b-pyramid.
    
• Your OBS parameters, seem strange:
  • vbv values will be ignored by x264 unless a level and profile is also configured. (this was years ago and afaik. this has never changed)
    
  • using opencl is a bad idea, since it will be slower and less quality than using the cpu (it basically was declared useless in the way it exists nowadays in x264 unless in some rare cases on 15+years old hardware)
    
  • 'filler' usually makes no sense unless you use some ancient editing or streaming software/hardware.
    
• NVEnc and b-pyramid with ffmpeg: '-b_ref_mode <mode>' is probably the value you want to use.
  iirc:
  The -b_ref_mode:v option in FFmpeg's NVENC encoder supports the following values:
  • disabled (0): B-frames will not be used for reference
    
  • each (1): Each B-frame will be used for reference
    
  • middle (2): Only (number of B-frames)/2 will be used for reference
    
  Also note that this is only supported for H.265 (hevc_nvenc), and like mentioned before for middle mode, you need at least -bf 2 (2 B-frames) and -refs 3 (3 reference frames).
  

Cu Selur

[BudgetGamer123]

Thank you for detailed and fast response dear @Selur
Indeed some meaningless variables there in old x264 preset and honestly I was never able to stream because of lacking connection...but what is really odd is popular streaming services using fixed GOP size. It has to be 2 seconds for twitch for example. Limiting 60fps stream to 120 Keyint, 30fps to 60 keyint. No other way around it. Which is why I had to dive into all this encoder technology in the first place as noob, trying to squeeze the max. amount of compression to that small, limited GOP Yeah adding OpenCL made quality worse, but at that time It was the only way to take some load off from CPU (It was overloading, and I didn't wanted to increase threads since it ruined quality further) So had to make that decision there.
At some point, I even read the intraRefresh parts of source code of x264 thinking maybe I can do it somehow with 2 second GOP limit and have a better distribution of data, and eliminate pulsation of quality with every I frame, then losing it all after some frames, then repeat. No luck there either.

So best tweak I could find was the strict B-Pyramid trick, with 64 fps. Even a naked eye comparison, it is still visible. Back then I did some VMAF tests on lossless in-game footage and saw it working great with measureable means as well in terms of compression.

For NVencoder tests I wanted to use FFMpeg Advanced tab in OBS but couldn't figure out why video encoder commands are not passing. Even prompting rate control as CBR and limiting bitrate doesn't work using commands such as:
rc=cbr maxBitrate=1000 averageBitRate=1000 vbvBufferSize=1000
and this doesn't pass to encoder settings.

From regular tab I have tested command you gave
[obs-nvenc: 'advanced_video_recording'] settings:
codec:        H264
rate_control: CQVBR
max_bitrate:  6000
cq:          23
keyint:      128
preset:      p7
tuning:      hq
multipass:    fullres
profile:      high
width:        1920
height:      1080
b-frames:    4
b-ref-mode:  0
lookahead:    true (31 frames)
aq:          true
user opts:    useBFramesAsRef=1 frameIntervalP=3 lookaheadDepth=31

added other two more options,
From what Ive read https://obsproject.com/kb/advanced-nvenc-options frameInterval setting changes the way how GOP starts and without any measurement, looks better at setting 2 and 3  gop patterns. how do I know with no measurement but some text looks less degraded with it, seems to have positive impact on data distribution. Not sure ofcourse. Again Thanks for reply. I will now have to figure out why settings at advanced ffmpeg tab doesn't pass to encoder. Always confusing. I don't get it. Should be this code:
https://www.ffmpeg.org/doxygen/trunk/nvE...ource.html

passing into encoder in ffmpeg right? why it doesn't work
EDIT:aaaaahhh... OBS ffmpeg option input expects a unit like M or k at end of bitrate definition.. like averageBitrate=500k it works. my bad