Is "--vbr 100" what you wanted? 100kbit/s seems way too low for 1920x1080.
Code:
OS Version Windows 10 x64 (19044) [UTF-8]
CPU AMD Ryzen 5 5600X 6-Core Processor [4.67GHz] (6C/12T)
GPU #0: NVIDIA GeForce GTX 1650 SUPER (1280 cores, 1725 MHz)[PCIe3x16][531.79]
NVENC / CUDA NVENC API 12.0, CUDA 12.1, schedule mode: auto
Input Buffers CUDA, 44 frames
Input Info y4m(yv12(10bit))->p010 [AVX2], 1920x1080, 60/1 fps
Vpp Filters copyHtoD
ssim psnr (yv12(10bit))
Output Info H.265/HEVC main10 @ Level auto
1920x1080p 1:1 60.000fps (60/1fps)
Encoder Preset default
Rate Control VBR
Multipass 2pass-quarter
Bitrate 100 kbps (Max: 800000 kbps)
Target Quality auto
Initial QP I:20 P:23 B:25
QP Offset cb:0 cr:0
VBV buf size auto
Lookahead on, 32 frames, Adaptive I Insert
GOP length 600 frames
B frames 3 frames [ref mode: middle]
Ref frames 3 frames, MultiRef L0:auto L1:auto
AQ off
CU max / min 32 / 8
VUI matrix:bt2020nc,range:limited
Others mv:Q-pel
64 frames: 99.22 fps, 17037 kb/s, GPU 3%, VE 0%
193 frames: 133.38 fps, 12354 kb/s, GPU 20%, VE 99%, VD 10%
322 frames: 143.17 fps, 11408 kb/s, GPU 20%, VE 99%, VD 10%
451 frames: 147.72 fps, 11188 kb/s, GPU 20%, VE 100%, VD 10%
581 frames: 150.64 fps, 10081 kb/s, GPU 20%, VE 99%, VD 10%
711 frames: 152.48 fps, 8757 kb/s, GPU 20%, VE 99%, VD 10%
841 frames: 153.83 fps, 8070 kb/s, GPU 20%, VE 99%, VD 10%
970 frames: 154.75 fps, 8764 kb/s, GPU 20%, VE 99%, VD 11%
1099 frames: 155.38 fps, 9056 kb/s, GPU 20%, VE 99%, VD 11%
1228 frames: 155.92 fps, 9305 kb/s, GPU 20%, VE 100%, VD 10%
1357 frames: 156.37 fps, 9316 kb/s, GPU 20%, VE 99%, VD 10%
1486 frames: 156.75 fps, 9561 kb/s, GPU 20%, VE 99%, VD 10%
1615 frames: 157.04 fps, 9721 kb/s, GPU 20%, VE 100%, VD 10%
1745 frames: 157.38 fps, 10027 kb/s, GPU 20%, VE 100%, VD 11%
on_stopAllPushButton_clicked
but uses really high bit rates.
The calls itself seem to be okay.