Selur's Little Message Board
[HELP] Unable to encode with NVEnC - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: [HELP] Unable to encode with NVEnC (/thread-4206.html)

Unable to encode with NVEnC - acusta - 06.09.2025

Hello Selur & others...

I recently upgraded my GPU to a RTX 5070. Latest drivers (game ready & studio) are installed.

I've been trying to encode using my GPU through NVenC but the job always crashes when starting.

Things i've done so far:

-Updated NvenC to version 9.0 

-Cleared cache tool info after copying and renaming NVEnCx64.exe to NVEnC.exe. (I also copied the rest of .dll from the latest NVEnC version into 64bit folder in Hybrid installation)

-Tool versions detects NVEnC as version 9

Jobs will always crash if "Use only encoder" checkbox is active, whatever the setting in NVEnC is. If not checked, the encoding will work but at my CPU speed (arround 165fps) and using 100% of the CPU and not even a 5% of my GPU.

I also tried checking the "Use gpu for decoding" box with no luck.

With my older GTX 1060, it used to encode at arround 350fps.

I'm attaching debug and report.

Thank you!


Report

Debug

RE: Unable to encode with NVEnC - Selur - 06.09.2025

AFK atm try latest dev, will test tomorrow

RE: Unable to encode with NVEnC - Selur - 07.09.2025

What format is the input in?

RE: Unable to encode with NVEnC - acusta - 07.09.2025

(07.09.2025, 08:57)Selur Wrote: What format is the input in?

Hello, the file is YUV V210 (10 bit uncompressed). I've spent a few hours reading some documentation and it seems that uncompressed video is not compatible with CUDA (at least through this method)

It seems funny, but if using NVIDIA as an encoder in Resolve and inputting the same file in order to reencode into MP4, it will work with an estimate of 500fps. This got me confused.

Thank you!

RE: Unable to encode with NVEnC - Selur - 07.09.2025

Yup, with 'only use encoder' Hybrid will tell NVEncC to use the hardware decoder chip which can only decode some stuff.
For some formats NVEncC will automatically switch to software decoding, but especially uncompressed usually is not supported.
Resolve uses software decoding. Smile
Side note: Software encoding can be faster depending on your hardware. Wink

Cu Selur

RE: Unable to encode with NVEnC - acusta - 07.09.2025

(07.09.2025, 11:22)Selur Wrote: Yup, with 'only use encoder' Hybrid will tell NVEncC to use the hardware decoder chip which can only decode some stuff.
For some formats NVEncC will automatically switch to software decoding, but especially uncompressed usually is not supported.
Resolve uses software decoding. Smile
Side note: Software encoding can be faster depending on your hardware. Wink

Cu Selur

Thank you Selur,

Is there a way I can speed up encodings using both CPU and GPU ? I'm actually converting this file to ProRes422Proxy + QTGMC(faster)+BOB and getting arround 120fps.

My PC specs are:

-Intel I5 12400F (2.5Ghz)
-32 GB RAM DDR5 3200
-RTX 5070 12GB

Both file input and outpur are in the same NVME drive (WD SN850X)

Thanks again!

RE: Unable to encode with NVEnC - Selur - 07.09.2025

Since Windows has no ProRes hardware encoding or decoding and your source isn't in a format that is supported by hardware decoder, only way where you can benefit for the gpu is during filtering.
Enabling both 'OpenCL' and 'DFTTest2' might help a bit.

Cu Selur