NVEnc Preview

[Dan64]

Hello Selur,

   as you know NVEnc is an encoder that includes a lot of GPU optimized filters.
   It could be useful to have the possibility to view the preview of these filters.
   Unfortunately being included in the encoder is not possible to see them in the Vapoursynth preview window.

   A workaround could be to write a script that provide the output of NVEnc.

   I found few information on this topic in internet and the only one that I consider useful is in this post: https://forum.doom9.org/showthread.php?t=165771&page=203

   I tried to adapt the script to NVEnc and I wrote the following code

import vapoursynth as vs from vapoursynth import core import subprocess import ctypes NVEnc = r'E:\VideoTest\TestSubs\NVEncC64.exe' source_path=r'E:\VideoTest\TestSubs\TestSubs-1.mp4' # Loading Plugins core.std.LoadPlugin(path="D:/Programs/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll") # current color space: YUV420P10, bit depth: 10, resolution: 1920x800, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive # Loading E:\VideoTest\TestSubs\TestSubs-1.mp4 using LWLibavSource clip = core.lsmas.LWLibavSource(source="E:/VideoTest/TestSubs/TestSubs-1.mp4", format="YUV420P8", stream_index=0, cache=0, fpsnum=25, prefer_hw=0) #this clip is not not needed, just to get width and height # Setting detected color matrix (470bg). clip = core.std.SetFrameProps(clip, _Matrix=5) # Setting color transfer info (470bg), when it is not set clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5) # Setting color primaries info (BT.709), when it is not set clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=1) # Setting color range to TV (limited) range. clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1) # making sure frame rate is set to 25 clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive # adjusting output color from: YUV420P8 to YUV420P10 for x265Model clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited") # set output frame rate to 25fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1) clipB = core.std.BlankClip(clip) w = clip.width h = clip.height Ysize  = w * h UVsize = w * h//4 frame_len = w * h * 3 // 2 #YUV420 nvenc_filters = [ '--vpp-pmd apply_count=3,strength=100,threshold=100', '--vpp-unsharp radius=4,weight=0.5,threshold=10',                 '--vpp-smooth quality=4,qp=60,prec=fp32', '--vpp-tweak brightness=0.01,contrast=1.01,gamma=0.98,saturation=1.35,hue=-2,swapuv=false', '--vpp-deband range=15,sample=1,thre=15,dither=15,seed=1234,blurfirst=off,rand_each_frame=off', ] NVEnc_cmd = [NVEnc,                  '--avhw',                  '--input', source_path,                  '--lossless',                  '--output-format raw',                                  '--output -', # output pipe                  ] core.log_message(2,' '.join(NVEnc_cmd)) pipe = subprocess.Popen(NVEnc_cmd, stdout = subprocess.PIPE, bufsize=frame_len) def load_frame(n,f):     try:         vs_frame = f.copy()         for i, size in enumerate([Ysize, UVsize, UVsize]):             ctypes.memmove(vs_frame.get_write_ptr(i), pipe.stdout.read(size),  size)         pipe.stdout.flush()     except Exception as e:         raise ValueError(repr(e))        return vs_frame try:     clip = core.std.ModifyFrame(clip, clip, load_frame) except ValueError as e:     pipe.terminate()     print(e) clip.set_output()

   Unfortunately the result is not very good, I think the the problem is related to the wrong computation of:  Ysize, UVsize

    You can download the complete example here: https://filebin.net/sotms5i2chuhw6mo

    I hope that you are able to fix it.

Thanks,
Dan

[humanoid86]

  you're lucky =) Mr Selur loves vapoursynth and NVenc

[Selur]

The idea from _Al_ was to use ffmpegs filters by:
a. feeding raw video to ffmpeg
b. filter it through ffmpeg
c. output raw video from ffmpeg
d. capture that raw output and replace the memory space the unfiltered framed occupied with the raw output of ffmpeg.
This can work input&output resolution and format did not change.
You use:

-f,--output-format <string> set output format of output file. if format is not specified, output format will be guessed from output file extension. set "raw" for H.264/ES output.

with NVEncC, which instead of ffmpeg:

-vcodec rawvideo -pix_fmt yuv420p -f rawvideo

does not output RAW YUV420P8 content, but raw H.264 !
This will not work, what will happen is that you only replace parts of the original frame info with the compressed H.264 output, so you will get half broken frames.
To get this working, you would need to decode the raw H.264 frames back to raw yuv420p8.

Cu Selur

Ps.: moved the thread

[Dan64]

Hello Selur,

   If this can help, it is possible pass the output of NVEnc to ffmepg, with this change

NVEnc_cmd = [NVEnc, '--avhw', '--input', source_path, '--lossless', '--output-format raw', '--output -', # output pipe                 '|',                 ffmpeg, '-i -', '-vcodec', 'rawvideo', '-pix_fmt', 'yuv420p',  '-f', 'rawvideo', '-'                 ]

   But the out put is still the same

   Do you have any Idea ?

Thanks,
Dan

[Selur]

No clue.

[Dan64]

I tried to dump the output of the 2 process.

The output of: "NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --output-format raw --output -" , is

 

Video ID                          : 1 Format                      : AVC Format/Info                : Advanced Video Codec Format profile              : High@L3 Format settings            : CABAC / 3 Ref Frames Format settings, CABAC      : Yes Format settings, Reference  : 3 frames Codec ID                    : V_MPEG4/ISO/AVC Duration                    : 2 min 2 s Bit rate                    : 571 kb/s Width                      : 720 pixels Height                      : 300 pixels Display aspect ratio        : 2.40:1 Frame rate mode            : Constant Frame rate                  : 25.000 FPS Color space                : YUV Chroma subsampling          : 4:2:0 Bit depth                  : 8 bits Scan type                  : Progressive Bits/(Pixel*Frame)          : 0.106 Stream size                : 8.30 MiB (100%)

having removed the option "--lossless" the output is much smaller.

The output of: "ffmpeg.exe -i E:\VideoTest\TestSubs\Test.mkv -vcodec h264 -pix_fmt yuv420p -f rawvideo -" , is

Video ID                          : 1 Format                      : AVC Format/Info                : Advanced Video Codec Format profile              : High@L3 Format settings            : CABAC / 4 Ref Frames Format settings, CABAC      : Yes Format settings, Reference  : 4 frames Codec ID                    : V_MPEG4/ISO/AVC Duration                    : 2 min 2 s Bit rate                    : 373 kb/s Width                      : 720 pixels Height                      : 300 pixels Display aspect ratio        : 2.40:1 Frame rate mode            : Constant Frame rate                  : 25.000 FPS Color space                : YUV Chroma subsampling          : 4:2:0 Bit depth                  : 8 bits Scan type                  : Progressive Bits/(Pixel*Frame)          : 0.069 Stream size                : 5.43 MiB (99%) Writing library            : x264 core 161 r3027 4121277 Encoding settings          : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 Default                    : Yes Forced                      : No

 
So it seems that the output is the same:

Codec ID                    : V_MPEG4/ISO/AVC
Frame rate                  : 25.000 FPS
Color space                : YUV
Chroma subsampling  : 4:2:0
Bit depth                    : 8 bits
Scan type                  : Progressive 

But is not working.

[Selur]

Using:

F:\Hybrid\64bit\NVEncC.exe --avhw --input "C:\Users\Selur\Desktop\TestSubs-1.mp4" --lossless --output-format raw --output - | F:\Hybrid\64bit\ffmpeg.exe -i - -an -vcodec rawvideo -pix_fmt yuv420p -f rawvideo - | f:\Hybrid\64bit\ffplay.exe -f rawvideo -pixel_format yuv420p -video_size 720x300 -i -

(not inside Vapoursynth, but directly in a terminal.)
I see the output, but there are also some outputs:

bitstream: Error writing file. bitstream: Not enough disk space!

So the output seems to be correct (raw yvu420p), but there might be an issue with the reading in Python.

But is not working.

Top description.

----
Yes,

NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --output-format raw --output -

and

ffmpeg.exe -i E:\VideoTest\TestSubs\Test.mkv -vcodec h264 -pix_fmt yuv420p -f rawvideo -

both create MPEG-4 AVC video.

In https://forum.doom9.org/showthread.php?p...ost1925640

ffmpeg -i source_path -vcodec rawvideo -pix_fmt yuv420p -f rawvideo -

was used which creates raw yuv420p video.

Cu Selur

Ps.: you are aware that even if one gets this working, you won't be able to jump inside the clip etc. You can only go forward by one frame.

[Dan64]

Willing to understand what is necessary to do, I think that we need to obtain with NVEnc a video with "uncompressed" frames.
This the first issue, because with NVEnc, i obtain the following output

NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output E:\VideoTest\TestSubs\Test.raw                    -------------------------------------------------------------------------------- E:\VideoTest\TestSubs\Test.raw -------------------------------------------------------------------------------- NVEncC (x64) 7.31 (r2580) by rigaya, Aug 28 2023 13:07:12 (VC 1929/Win) OS Version    Windows 10 x64 (19045) [UTF-8] CPU            Intel Core i9-10900 @ 2.80GHz [TB: 4.82GHz] (10C/20T) GPU            #0: NVIDIA GeForce RTX 3060 (3584 cores, 1837 MHz)[PCIe3x16][537.13] NVENC / CUDA  NVENC API 12.1, CUDA 12.2, schedule mode: auto Input Buffers  CUDA, 20 frames Input Info    avcuvid: H.264/AVC, 720x300, 25/1 fps AVSync        vfr Vpp Filters    copyDtoD Output Info    H.264/AVC Unknown @ Level auto               720x300p 1:1 25.000fps (25/1fps) Encoder Preset default Rate Control  CQP  I:0  P:0  B:0 (lossless) ChromaQPOffset cb:0  cr:0 Split Enc Mode auto Lookahead      off GOP length    250 frames B frames      3 frames [ref mode: disabled] Ref frames    3 frames, MultiRef L0:auto L1:auto AQ            off Others        mv:auto cabac deblock adapt-transform:auto bdirect:auto encoded 3050 frames, 1034.95 fps, 9176.35 kbps, 133.46 MB encode time 0:00:02, CPU: 3.1%, GPU: 15.0%, VE: 49.7%, VD: 27.0%, GPUClock: 1890MHz, VEClock: 1665MHz frame type IDR  13 frame type I    13,  total size  1.01 MB frame type P    769,  total size  40.79 MB frame type B  2268,  total size  91.65 MB

The frames are compressed " H.264/AVC Unknown @ Level auto ", this fact is confirmed by Mediainfo

Video ID                          : 1 Format                      : AVC Format/Info                : Advanced Video Codec Format profile              : High 4:4:4 Predictive@L3 Format settings            : CABAC / 3 Ref Frames Format settings, CABAC      : Yes Format settings, Reference  : 3 frames Codec ID                    : V_MPEG4/ISO/AVC Duration                    : 2 min 2 s Bit rate                    : 9 176 kb/s Width                      : 720 pixels Height                      : 300 pixels Display aspect ratio        : 2.40:1 Frame rate mode            : Constant Frame rate                  : 25.000 FPS Color space                : YUV Chroma subsampling          : 4:2:0 Bit depth                  : 8 bits Scan type                  : Progressive Bits/(Pixel*Frame)          : 1.699 Stream size                : 133 MiB (100%) Default                    : Yes Forced                      : No

This is the first issue. I tried to fix the problem by using ffmpeg that is able to provide in output "true" raw video, but I think that the second pipe is "frozing" the process because I'm unable to forward the frames, while this is possible by using only ffmpeg, even if in this case the colors are wrong.

So the question is, it is possible to use NVEnc to produce in output "true" uncompressed frames ?

---

While using "ffmpeg" in pipe I obtaing the following output

NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output -| ffmpeg.exe -i - -vcodec rawvideo -pix_fmt yuv420p -loglevel debug -f rawvideo E:\VideoTest\TestSubs\Test2.raw -------------------------------------------------------------------------------- - -------------------------------------------------------------------------------- NVEncC (x64) 7.31 (r2580) by rigaya, Aug 28 2023 13:07:12 (VC 1929/Win) OS Version    Windows 10 x64 (19045) [UTF-8] CPU            Intel Core i9-10900 @ 2.80GHz [TB: 4.70GHz] (10C/20T) GPU            #0: NVIDIA GeForce RTX 3060 (3584 cores, 1837 MHz)[PCIe3x16][537.13] NVENC / CUDA  NVENC API 12.1, CUDA 12.2, schedule mode: auto Input Buffers  CUDA, 20 frames Input Info    avcuvid: H.264/AVC, 720x300, 25/1 fps AVSync        vfr Vpp Filters    copyDtoD Output Info    H.264/AVC Unknown @ Level auto               720x300p 1:1 25.000fps (25/1fps) Encoder Preset default Rate Control  CQP  I:0  P:0  B:0 (lossless) ChromaQPOffset cb:0  cr:0 Split Enc Mode auto Lookahead      off GOP length    250 frames B frames      3 frames [ref mode: disabled] Ref frames    3 frames, MultiRef L0:auto L1:auto AQ            off Others        mv:auto cabac deblock adapt-transform:auto bdirect:auto encoded 3050 frames, 767.88 fps, 9176.35 kbps, 133.46 MB encode time 0:00:03, CPU: 1.4%, GPU: 8.6%, VE: 34.2%, VD: 20.6%, GPUClock: 1837MHz, VEClock: 1612MHz frame type IDR  13 frame type I    13,  total size  1.01 MB frame type P    769,  total size  40.79 MB frame type B  2268,  total size  91.65 MB ffmpeg version N-100400-g5f7dc836ce-gd76469378d+4 Copyright (c) 2000-2020 the FFmpeg developers   built with gcc 10.2.0 (Rev5, Built by MSYS2 project)   configuration:  --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libaom --enable-libopenmpt --enable-version3 --enable-libsvthevc --enable-libsvtav1 --enable-libkvazaar --enable-libzmq --enable-libvmaf --enable-vapoursynth --disable-libdavs2 --enable-librav1e --enable-mbedtls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DZMQ_STATIC --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --extra-cflags=-DKVZ_STATIC_LIB   libavutil      56. 62.100 / 56. 62.100   libavcodec    58.115.102 / 58.115.102   libavformat    58. 65.100 / 58. 65.100   libavdevice    58. 11.103 / 58. 11.103   libavfilter    7. 93.100 /  7. 93.100   libswscale      5.  8.100 /  5.  8.100   libswresample  3.  8.100 /  3.  8.100   libpostproc    55.  8.100 / 55.  8.100 Splitting the commandline. Reading option '-i' ... matched as input url with argument '-'. Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'rawvideo'. Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'. Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'. Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'. Reading option 'E:\VideoTest\TestSubs\Test2.raw' ... matched as output url. Finished splitting the commandline. Parsing a group of options: global . Applying option loglevel (set logging level) with argument debug. Successfully parsed a group of options. Parsing a group of options: input url -. Successfully parsed a group of options. Opening an input file: -. [NULL @ 0000023ee2258500] Opening 'pipe:' for reading [pipe @ 0000023ee2259600] Setting default whitelist 'crypto,data' [AVIOContext @ 0000023ee2259800] Statistics: 1048576 bytes read, 0 seeks pipe:: Invalid data found when processing input

So it seems that the pipe process is broken, and this could explain the frozen process.

[Selur]

So the question is, it is possible to use NVEnc to produce in output "true" uncompressed frames ?

No, NVEncC, can't output uncompressed frames.

NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output -| ffmpeg.exe -i - -vcodec rawvideo -pix_fmt yuv420p -loglevel debug -f rawvideo E:\VideoTest\TestSubs\Test2.raw

That does not work, since you output raw H.264, but tell ffmpeg that it reads raw uncompressed video with an unknown size.

NVEncC64.exe --avhw --inputE:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output - | ffmpeg.exe -i - -an -vcodec rawvideo -pix_fmt yuv420p -f rawvideo Test_raw_yuv420p.raw

will output a raw yuv420 file.

[Dan64]

I opened an issue with Rigaya: https://github.com/rigaya/NVEnc/issues/5...1920845626