4K transcoding problem with NVENC (outdated false limitation in Hybrid)

[No limit]

• Please, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.
  

Hello!

I read the previous article: https://forum.selur.net/showthread.php?tid=80

I have a Geforce GTX 1060 card in my PC, my father had an 1050TI, both Card support 4K transcoding with NVENC, however Hybrid does not allow the transcoding of 4K videos with the normal NVENC settings, because of MAX MB per sec limitation was automatically set to Hybrid. It is however not correct for Nvidia Pascal Geforce 1050 1060 1070 1080 series of cards, especially after the new drivers.

I could transcode 4K videos with NVENC on my computer and on my father's computer without any problems in Staxrip, MEdiacoder, Xmedia Recode, nvenc supporting plugins for Premiere Pro and other softwares. So it is clear a proof, that there is no limit on these Geforce cards and new drivers in the case of 4K videos.

Thank you!

[Selur]

I tried to reproduce this with my 1070ti, by:

• Starting Hybrid
  
• Loading a 4k source
  
• Selecting NVEnc as encoder (Base->Processing->Video->NVEnc)
  
• Settings 'NVEnc->Base->Encoding mode' to 'constant quantizer'
  Command line looks like this:
  
  NVEnc --y4m -i - --fps 25.000 --codec h264 --profile high --level auto --sar 1:1 --lookahead 16 --cqp 20:23:25 --gop-len 0 --bframes 3 --ref 3 --mv-precision Q-pel --cabac --deblock --preset default --videoformat undef --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --output "E:\Temp\4K sun HDR test.264"
• Setting an output file (Base->Output file)
  
• Creating a new job (Base->Add to queue)
  
• Starting the job (Jobs->Queue->Start)
  

-> Encoding worked fine.

Only options that could be possible to limit the max MBs per second are the profile&level settings.
Other than that I got no clue what settings you used.
My current guess is that you used a profile&level combination which doesn't allow the bitrate+fps+resolution combination you chose.

=>

Please, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.

Without proper detail there is no way for me to reproduce the issue.


Cu Selur

[No limit]

I tried to reproduce this with my 1070ti, by:

• Starting Hybrid
  
• Loading a 4k source
  
• Selecting NVEnc as encoder (Base->Processing->Video->NVEnc)
  
• Settings 'NVEnc->Base->Encoding mode' to 'constant quantizer'
  Command line looks like this:
  NVEnc --y4m -i - --fps 25.000 --codec h264 --profile high --level auto --sar 1:1 --lookahead 16 --cqp 20:23:25 --gop-len 0 --bframes 3 --ref 3 --mv-precision Q-pel --cabac --deblock --preset default --videoformat undef --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --output "E:\Temp\4K sun HDR test.264"
  
• Setting an output file (Base->Output file)
  
• Creating a new job (Base->Add to queue)
  
• Starting the job (Jobs->Queue->Start)
  

-> Encoding worked fine.

Only options that could be possible to limit the max MBs per second are the profile&level settings.
Other than that I got no clue what settings you used.
My current guess is that you used a profile&level combination which doesn't allow the bitrate+fps+resolution combination you chose.

=>

Please, read the 'Infos needed to fix&reproduce bugs,..'-sticky before you post about a problem.

Without proper detail there is no way for me to reproduce the issue.


Cu Selur

Level is always "Auto", but I tried with 6.2, but  nothing has changed.

All other settings are "factory default"

Sometimes it can start, but automatically revert the output to 1080p.

Other above mentioned transcoder softwares can do 4K NVENC HEVC 10bit videos without any problems & warnings.

[Selur]

What does:

NVEncC --check-features

output on your system?

I get for example:

Environment Info
OS : Windows 10 x64 (17134)
CPU: AMD Ryzen 7 1800X Eight-Core Processor (8C/16T)
RAM: Used 5469 MB, Total 32696 MB
GPU: GeForce GTX 1070 Ti (19 EU) @ 1683 MHz (399.07)

List of available features.
Codec: H.264/AVC
Max Bframes               4
B Ref Mode                yes
RC Modes                  63
Field Encoding            yes
MonoChrome                no
FMO                       no
Quater-Pel MV             yes
B Direct Mode             yes
CABAC                     yes
Adaptive Transform        yes
Max Temporal Layers       0
Hierarchial P Frames      no
Hierarchial B Frames      no
Max Level                 51
Min Level                 1
4:4:4                     yes
Max Width                 4096
Max Height                4096
Dynamic Resolution Change yes
Dynamic Bitrate Change    yes
Forced constant QP        yes
Dynamic RC Mode Change    no
Subframe Readback         yes
Constrained Encoding      yes
Intra Refresh             yes
Custom VBV Bufsize        yes
Dynamic Slice Mode        yes
Ref Pic Invalidiation     yes
PreProcess                no
Async Encoding            yes
Max MBs                   65536
MAX MB per sec            983040
Lossless                  yes
SAO                       no
Me Only Mode              yes
Lookahead                 yes
AQ (temporal)             yes
Weighted Prediction       yes
Max LTR Frames            8
10bit depth               no

Codec: H.265/HEVC
Max Bframes               0
RC Modes                  63
Field Encoding            no
MonoChrome                no
Quater-Pel MV             yes
B Direct Mode             no
Max Temporal Layers       0
Hierarchial P Frames      no
Hierarchial B Frames      no
Max Level                 62
Min Level                 1
4:4:4                     yes
Max Width                 8192
Max Height                8192
Dynamic Resolution Change yes
Dynamic Bitrate Change    yes
Forced constant QP        yes
Dynamic RC Mode Change    no
Subframe Readback         yes
Constrained Encoding      no
Intra Refresh             yes
Custom VBV Bufsize        yes
Dynamic Slice Mode        yes
Ref Pic Invalidiation     yes
PreProcess                no
Async Encoding            yes
Max MBs                   262144
MAX MB per sec            983040
Lossless                  yes
SAO                       yes
Me Only Mode              yes
Lookahead                 yes
AQ (temporal)             no
Weighted Prediction       yes
Max LTR Frames            7
10bit depth               yes

So on my system the max MB count per second is restricted to 983040 which seems to be the same on your system by the error popup.
Hybrid calculates the MBs per secund through:

MAX MB per sec = (width / 16.0 + 0.5) * (height / 16.0 + 0.5) * fps

which would be in your case:
3840/16 * 2140/16 * 59.94 = 1 924 074
which is why Hybrid aborts here.
(983040 would be enough for 3840/16 * 2140/16 * 30 = 963000)

Grabbing the latest NVEncC 4.14 from the authors GoogleDrive, extracting the content of the 'NVEncC\64x'-folder into the 'Hybrid/64bit'-folder and renaming NVEncC64.exe to NVEncC.exe and running the call from above again doesn't change the output for me.

So there are multiple scenarios:
a. I/Hybrid calculates the macro blocks per second wrong
b. the value NVEncC output wrong data
->
a. if the calculation is wrong, I would need to know how to calculate this correctly (I don't see a mistake here since macro blocks should be 16x16 pixel in size)
b. If the restriction is really outdated, this needs to be fixed in NVEncC which Hybrid uses. Once this is fixed in NVEncC replacing the file like mentioned above and calling 'Config->Tools->Clear all cached tool infos' will fix the issue. In case this is a bug in NVEncCs output, I do not plan to write a workaround for this, since I believe this should be fixed in NVEncC then.

Cu Selur

Ps.: NVEncCs 'home' is https://github.com/rigaya/NVEnc

[No limit]

What does:

NVEncC --check-features

output on your system?

I get for example:

Environment Info
OS : Windows 10 x64 (17134)
CPU: AMD Ryzen 7 1800X Eight-Core Processor (8C/16T)
RAM: Used 5469 MB, Total 32696 MB
GPU: GeForce GTX 1070 Ti (19 EU) @ 1683 MHz (399.07)

List of available features.
Codec: H.264/AVC
Max Bframes               4
B Ref Mode                yes
RC Modes                  63
Field Encoding            yes
MonoChrome                no
FMO                       no
Quater-Pel MV             yes
B Direct Mode             yes
CABAC                     yes
Adaptive Transform        yes
Max Temporal Layers       0
Hierarchial P Frames      no
Hierarchial B Frames      no
Max Level                 51
Min Level                 1
4:4:4                     yes
Max Width                 4096
Max Height                4096
Dynamic Resolution Change yes
Dynamic Bitrate Change    yes
Forced constant QP        yes
Dynamic RC Mode Change    no
Subframe Readback         yes
Constrained Encoding      yes
Intra Refresh             yes
Custom VBV Bufsize        yes
Dynamic Slice Mode        yes
Ref Pic Invalidiation     yes
PreProcess                no
Async Encoding            yes
Max MBs                   65536
MAX MB per sec            983040
Lossless                  yes
SAO                       no
Me Only Mode              yes
Lookahead                 yes
AQ (temporal)             yes
Weighted Prediction       yes
Max LTR Frames            8
10bit depth               no

Codec: H.265/HEVC
Max Bframes               0
RC Modes                  63
Field Encoding            no
MonoChrome                no
Quater-Pel MV             yes
B Direct Mode             no
Max Temporal Layers       0
Hierarchial P Frames      no
Hierarchial B Frames      no
Max Level                 62
Min Level                 1
4:4:4                     yes
Max Width                 8192
Max Height                8192
Dynamic Resolution Change yes
Dynamic Bitrate Change    yes
Forced constant QP        yes
Dynamic RC Mode Change    no
Subframe Readback         yes
Constrained Encoding      no
Intra Refresh             yes
Custom VBV Bufsize        yes
Dynamic Slice Mode        yes
Ref Pic Invalidiation     yes
PreProcess                no
Async Encoding            yes
Max MBs                   262144
MAX MB per sec            983040
Lossless                  yes
SAO                       yes
Me Only Mode              yes
Lookahead                 yes
AQ (temporal)             no
Weighted Prediction       yes
Max LTR Frames            7
10bit depth               yes

So on my system the max MB count per second is restricted to 983040 which seems to be the same on your system by the error popup.
Hybrid calculates the MBs per secund through:

MAX MB per sec = (width / 16.0 + 0.5) * (height / 16.0 + 0.5) * fps

which would be in your case:
3840/16 * 2140/16 * 59.94 = 1 924 074
which is why Hybrid aborts here.

Grabbing the latest NVEncC 4.14 from the authors GoogleDrive, extracting the content of the 'NVEncC\64x'-folder into the 'Hybrid/64bit'-folder and renaming NVEncC64.exe to NVEncC.exe and running the call from above again doesn't change the output for me.

So there are multiple scenarios:
a. I/Hybrid calculates the macro blocks per second wrong
b. the value NVEncC output wrong data
->
a. if the calculation is wrong, I would need to know how to calculate this correctly (I don't see a mistake here since macro blocks should be 16x16 pixel in size)
b. If the restriction is really outdated, this needs to be fixed in NVEncC which Hybrid uses. Once this is fixed in NVEncC replacing the file like mentioned above and calling 'Config->Tools->Clear all cached tool infos' will fix the issue.

Cu Selur

That's the problem with the over automatizing to prevent every possible failure, and the software won't start correctly. Can you switch off this feature for GF 1000 series, because with the new drivers these cards had not such problems during 4K transcoding, which means that the calculation is somehow wrong in the Hybrids error-message. 

Meanwhile I tried 8K NVENC transcoding in Staxrip, and everything works fine. GF 1000 series with Pascal support have a max encoding limit in 8K resolution. So in the case of Pascal driven Cards, you can turn off these thing. Okay, people who use old VGA drivers can have problems, but new drivers fixed that Issue from 2018.

Here is the NVENC support matrix (Encode decode everything.)  https://developer.nvidia.com/video-encod...ort-matrix

And it has right, 8K encoding works with Pascal engines.

[Selur]

-> Won't change this in Hybrid, but I'll post into the NVEncC bug tracker.

[No limit]

-> Won't change this in Hybrid, but I'll post into the NVEncC bug tracker.

Every GF 1000 Cards have the very same NVENC Chip and they have the same speed. 1070-1080 cards have 2 NVENC chips, but you can use the two chip, if you transcode 2 videos in the same time, or making two video streaming in the same time.

If you transcode or stream only 1 video, 1070-1080 is not faster.

Here is a short video, and test about it:
It is worth to watch:




Think it over in logical way. That error message and restriction is not needed. Every PASCAL CHIP IS THE SAME, and all of them are capable for 8K encoding.

The other above mentioned softwares don't care about such message, and they just simply work fine without (imagined fantasy) restrictions, false error messages and problems. Again: It is not needed anymore. IT is so simple like 2X2 =4

[Selur]

Posted to the rigayas bug tracker (https://github.com/rigaya/NVEnc/issues/71).

If you can't wait till he fixes it in NVEncC, simply don't use Hybrid but any of the software you mentioned before.
Like I wrote before:

I do not plan to write a workaround for this, since I believe this should be fixed in NVEncC then.

(you probably missed that edit since you were writing your answer at the time)

Cu Selur

[No limit]

Posted to the rigayas bug tracker (https://github.com/rigaya/NVEnc/issues/71).

If you can't wait till he fixes it in NVEncC, simply don't use Hybrid but any of the software you mentioned before.
Like I wrote before:

I do not plan to write a workaround for this, since I believe this should be fixed in NVEncC then.

(you probably missed that edit since you were writing your answer at the time)

Cu Selur

If you like the perfect solutions, than you can ask them, that they can concentrate on the 8K videos too, because they are working fine with PASCAL chips. Okay?
(Maybe You can modify your post on Github)

[Selur]

-> after feedback from rigaya I'll remove the MB check