[BUG] NVenc (FFmpeg) x265 output is scrambled

[Darkyere]

• What Hybrid version did you use?(Windows/Mac/Linux 32/64bit + versions number)
  Hybrid_2020.05.01.1_SETUP.exe (Windows)
  
• What did happen? If an error message came up what did it exactly say?
  The video gets compiled with no errors - but the output is two images at once with wrong colors and size
  
• What did you do to get to the bug? (best thing is a detailed step-by-step guide so that I can reproduce the problem)
  Base->Video=NVenc (FFmpeg) / Base-> Audio=auto add (All) / NVenc(FFmpeg)->Encoder=H265 / Preset=Blu Ray / Profile=Main / Level=Auto / Tier=High / Mode=1-pass variable bitrate (2pass optimization) - Bitrate=20000
  
• Debug output would be helpful:
  
  HybridDebugOutput.7z (23,54 KB)[attachment=897]
  
• a small sample, which allows the reprocudtion of the problem would be nice
  

         I would like to give sample input but the input is 120mb and i cant upload it beacuse that format is not supported (mp4). IF i 7z it is to big (Max Allowed 20mb and it is 120)

So here is MediaInfo information on the input [attachment=898]

         I would like to give sample output but the output is 47mb and i cant upload it beacuse that format is not supported  (mp4). IF i 7z it is to big (Max Allowed 20mb and it is 120)

So here is MediaInfo information on output [attachment=899]

----

As an extra note input is generated with  Nvidia ShadowPlay recording at 50mbps.
NVenc (FFmpeg) with x264 and same settings has no problem.
Output is being played with VLC media player 3.0.10

----

Hope its enough info

Best Regards,
Darkyere

[Selur]

Since I'm planning to remove 'NVEnc (FFmpeg)' from Hybrid for Windows, please try whether this works fine if you use 'NVEnc' instead. (It uses the same Chip&Library as NVEnc, but is poorly documented so it does not really make sense to keep it.)

Cu Selur

[Darkyere]

Thank you this solved my issue. This works without a hint of video or audio issues.

-

But there dont seem to be a lot of options.

Like "slow preset" which should be better quality than "Quality setting".

Or something like 2-pass encoding.

Is this something that is possible in the future ?

[Selur]

You have to ask NVIDIA for those, NVEnc like NVEnc (FFmpeg) both only offer what NVIDIA puts in their chips and exposes through the SDK, but I doubt that real encoding 2pass or slower encoding will come any time some and definitely not for existing NVIDIA chips.

Cu Selur

[Darkyere]

Thank you for the responses.

I started testing this on a Gaming Computer, and dont intend to Encode video on it in the long run.

I have an Ubuntu 20.04 with GTX 1050 in it but i dont see the NVEnc (Without ffmpeg) in it.

Before taking that graphic card into a windows PC is there any possibility that NVEnc (Without ffmpeg) will be available for in Ubuntu or any other Linux Distro in general ?

[Selur]

Main problem atm. is that I don't find the time to look seriously into compiling NVEncC, writing support for it into Hybrid is easy.

[Darkyere]

I appreciate a lot all the help u have given me.

And i of course understand it makes more sense for me to change the two Graphic cards. 
Then you having to fight through a long line of compiling issues getting it to work.

So i switched them yesterday and it works marvelously.

----

Just to check.
Handbrake which uses FFmpeg to encode can use the slow preset which should be better than the Quality preset and that preset is 2-pass.
Here is a show of the available options for FFmpeg (NVEnc)

ffmpeg -hide_banner -h encoder=hevc_nvenc | xclip -sel clip

Encoder hevc_nvenc [NVIDIA NVENC hevc encoder]:
    General capabilities: delay
    Threading capabilities: none
    Supported pixel formats: yuv420p nv12 p010le yuv444p yuv444p16le bgr0 rgb0 cuda
hevc_nvenc AVOptions:
  -preset            <int>        E..V.... Set the encoding preset (from 0 to 11) (default medium)
     default                      E..V....
     slow                         E..V.... hq 2 passes
     medium                       E..V.... hq 1 pass
     fast                         E..V.... hp 1 pass
     hp                           E..V....
     hq                           E..V....
     bd                           E..V....
     ll                           E..V.... low latency
     llhq                         E..V.... low latency hq
     llhp                         E..V.... low latency hp
     lossless                     E..V.... lossless
     losslesshp                   E..V.... lossless hp
  -profile           <int>        E..V.... Set the encoding profile (from 0 to 4) (default main)
     main                         E..V....
     main10                       E..V....
     rext                         E..V....
  -level             <int>        E..V.... Set the encoding level restriction (from 0 to 186) (default auto)
     auto                         E..V....
     1                            E..V....
     1.0                          E..V....
     2                            E..V....
     2.0                          E..V....
     2.1                          E..V....
     3                            E..V....
     3.0                          E..V....
     3.1                          E..V....
     4                            E..V....
     4.0                          E..V....
     4.1                          E..V....
     5                            E..V....
     5.0                          E..V....
     5.1                          E..V....
     5.2                          E..V....
     6                            E..V....
     6.0                          E..V....
     6.1                          E..V....
     6.2                          E..V....
  -tier              <int>        E..V.... Set the encoding tier (from 0 to 1) (default main)
     main                         E..V....
     high                         E..V....
  -rc                <int>        E..V.... Override the preset rate-control (from -1 to INT_MAX) (default -1)
     constqp                      E..V.... Constant QP mode
     vbr                          E..V.... Variable bitrate mode
     cbr                          E..V.... Constant bitrate mode
     vbr_minqp                    E..V.... Variable bitrate mode with MinQP (deprecated)
     ll_2pass_quality              E..V.... Multi-pass optimized for image quality (deprecated)
     ll_2pass_size                E..V.... Multi-pass optimized for constant frame size (deprecated)
     vbr_2pass                    E..V.... Multi-pass variable bitrate mode (deprecated)
     cbr_ld_hq                    E..V.... Constant bitrate low delay high quality mode
     cbr_hq                       E..V.... Constant bitrate high quality mode
     vbr_hq                       E..V.... Variable bitrate high quality mode
  -rc-lookahead      <int>        E..V.... Number of frames to look ahead for rate-control (from 0 to INT_MAX) (default 0)
  -surfaces          <int>        E..V.... Number of concurrent surfaces (from 0 to 64) (default 0)
  -cbr               <boolean>    E..V.... Use cbr encoding mode (default false)
  -2pass             <boolean>    E..V.... Use 2pass encoding mode (default auto)
  -gpu               <int>        E..V.... Selects which NVENC capable GPU to use. First GPU is 0, second is 1, and so on. (from -2 to INT_MAX) (default any)
     any                          E..V.... Pick the first device available
     list                         E..V.... List the available devices
  -delay             <int>        E..V.... Delay frame output by the given amount of frames (from 0 to INT_MAX) (default INT_MAX)
  -no-scenecut       <boolean>    E..V.... When lookahead is enabled, set this to 1 to disable adaptive I-frame insertion at scene cuts (default false)
  -forced-idr        <boolean>    E..V.... If forcing keyframes, force them as IDR frames. (default false)
  -spatial_aq        <boolean>    E..V.... set to 1 to enable Spatial AQ (default false)
  -temporal_aq       <boolean>    E..V.... set to 1 to enable Temporal AQ (default false)
  -zerolatency       <boolean>    E..V.... Set 1 to indicate zero latency operation (no reordering delay) (default false)
  -nonref_p          <boolean>    E..V.... Set this to 1 to enable automatic insertion of non-reference P-frames (default false)
  -strict_gop        <boolean>    E..V.... Set 1 to minimize GOP-to-GOP rate fluctuations (default false)
  -aq-strength       <int>        E..V.... When Spatial AQ is enabled, this field is used to specify AQ strength. AQ strength scale is from 1 (low) - 15 (aggressive) (from 1 to 15) (default 8)
  -cq                <float>      E..V.... Set target quality level (0 to 51, 0 means automatic) for constant quality mode in VBR rate control (from 0 to 51) (default 0)
  -aud               <boolean>    E..V.... Use access unit delimiters (default false)
  -bluray-compat     <boolean>    E..V.... Bluray compatibility workarounds (default false)
  -init_qpP          <int>        E..V.... Initial QP value for P frame (from -1 to 51) (default -1)
  -init_qpB          <int>        E..V.... Initial QP value for B frame (from -1 to 51) (default -1)
  -init_qpI          <int>        E..V.... Initial QP value for I frame (from -1 to 51) (default -1)
  -qp                <int>        E..V.... Constant quantization parameter rate control method (from -1 to 51) (default -1)
  -weighted_pred     <int>        E..V.... Set 1 to enable weighted prediction (from 0 to 1) (default 0)

Are these preset's and options available for NVEncC so they could be implemented? Or is it someting FFMpeg have metteled with themselves ?

->

Because when trying to find official information about it when searching on NVEnc and going to Nvidia Developer SDK i cant seem to find 2 pass or slow mode (Even though FFMpeg and hadbrake seems to make use of such modes)

----

But ofcouse its out of my leagua i am just hunting the best way to mass convert a lot of Nvidia Shadowplay Highlights/Videos as fast as possible with no loss of quality.

8 core CPU gives me 8-10 FPS, but NVEnvC (With or Without FFMPEG) with 4 cores/GtX 1050 gives me 100-175FPS so its definetly the fastest i have ever used.

The only thing i am interested in, is to get the best quality option that is available to optimize the result for lower bitrate.

[Selur]

1st Handbrake does not use FFmpeg, it uses libav which is part of the FFmpeg project.
2nd what FFmpeg calls 'slow' is sometimes called 'hq 2 passes' what you get when you use 'variable bitrate (HQ)' in Hybrid. (+ may be encoder preset->quality)
NVEnc SDK does not offer real 2pass encoding (where you run two separated passes, 1st pass creating a statistic file and the second pass using that statistic file to make better decisions).
The '2pass' encoding NVEnc offers is an encoding where each frame gets encoded 2 times, where tries to optimize the encoding. This is similar to the 'real' 2pass encoding, but only similar. During 'real' 2pass encoding the decisions can be based on the data of all frames of the clip, wheres NVEncs decisions can only be based a previous encode of the current single frame (+ lookahead).
Since lots of folks don't like that NVEnc calls it's method 2pass encoding folks usually call it 'XY hq' or similar.

-> There is no real 2pass encoding like x264 and x265 offers when using NVEnc, doesn't matter if you use Handbrake, FFmpeg or whatever tool you like.

The only thing i am interested in, is to get the best quality option that is available to optimize the result for lower bitrate.

For lower bit rates better use software encoding, last I checked x264/x265 were noticeable better (but slower) there.


Cu Selur

[Darkyere]

Yeah i almost new it was more than what i was capable of reading up on.

I will definetly have a look at x264/x265 software encodng again.
I just felt like this would be the epic way to convert files to a more sutiable size. I didnt know it had so little options.

Thank you again for your time and best regards,
Darkyere