Selur's Little Message Board

Full Version: NVEnc Preview
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4
Hello Selur,

   as you know NVEnc is an encoder that includes a lot of GPU optimized filters.
   It could be useful to have the possibility to view the preview of these filters.
   Unfortunately being included in the encoder is not possible to see them in the Vapoursynth preview window.

   A workaround could be to write a script that provide the output of NVEnc.

   I found few information on this topic in internet and the only one that I consider useful is in this post: https://forum.doom9.org/showthread.php?t=165771&page=203

   I tried to adapt the script to NVEnc and I wrote the following code

Code:
import vapoursynth as vs
from vapoursynth import core
import subprocess
import ctypes

NVEnc = r'E:\VideoTest\TestSubs\NVEncC64.exe'
source_path=r'E:\VideoTest\TestSubs\TestSubs-1.mp4'
# Loading Plugins
core.std.LoadPlugin(path="D:/Programs/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# current color space: YUV420P10, bit depth: 10, resolution: 1920x800, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading E:\VideoTest\TestSubs\TestSubs-1.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="E:/VideoTest/TestSubs/TestSubs-1.mp4", format="YUV420P8", stream_index=0, cache=0, fpsnum=25, prefer_hw=0) #this clip is not not needed, just to get width and height
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer info (470bg), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (BT.709), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clipB = core.std.BlankClip(clip)

w = clip.width
h = clip.height
Ysize  = w * h
UVsize = w * h//4
frame_len = w * h * 3 // 2 #YUV420

nvenc_filters = [
'--vpp-pmd apply_count=3,strength=100,threshold=100',
'--vpp-unsharp radius=4,weight=0.5,threshold=10',                
'--vpp-smooth quality=4,qp=60,prec=fp32',
'--vpp-tweak brightness=0.01,contrast=1.01,gamma=0.98,saturation=1.35,hue=-2,swapuv=false',
'--vpp-deband range=15,sample=1,thre=15,dither=15,seed=1234,blurfirst=off,rand_each_frame=off',
]

NVEnc_cmd = [NVEnc,
                 '--avhw',
                 '--input', source_path,
                 '--lossless',
                 '--output-format raw',                
                 '--output -', # output pipe
                 ]

core.log_message(2,' '.join(NVEnc_cmd))

pipe = subprocess.Popen(NVEnc_cmd, stdout = subprocess.PIPE, bufsize=frame_len)

def load_frame(n,f):
    try:
        vs_frame = f.copy()
        for i, size in enumerate([Ysize, UVsize, UVsize]):
            ctypes.memmove(vs_frame.get_write_ptr(i), pipe.stdout.read(size),  size)
        pipe.stdout.flush()
    except Exception as e:
        raise ValueError(repr(e))   
    return vs_frame

try:
    clip = core.std.ModifyFrame(clip, clip, load_frame)
except ValueError as e:
    pipe.terminate()
    print(e)

clip.set_output()

   Unfortunately the result is not very good, I think the the problem is related to the wrong computation of:  Ysize, UVsize

    You can download the complete example here: https://filebin.net/sotms5i2chuhw6mo

    I hope that you are able to fix it.

Thanks,
Dan
Big Grin  you're lucky =) Mr Selur loves vapoursynth and NVenc
The idea from _Al_ was to use ffmpegs filters by:
a. feeding raw video to ffmpeg
b. filter it through ffmpeg
c. output raw video from ffmpeg
d. capture that raw output and replace the memory space the unfiltered framed occupied with the raw output of ffmpeg.
This can work input&output resolution and format did not change.
You use:
Code:
-f,--output-format <string>     set output format of output file.
                                 if format is not specified, output format will
                                 be guessed from output file extension.
                                 set "raw" for H.264/ES output.
with NVEncC, which instead of ffmpeg:
Code:
-vcodec rawvideo -pix_fmt yuv420p -f rawvideo
does not output RAW YUV420P8 content, but raw H.264 !
This will not work, what will happen is that you only replace parts of the original frame info with the compressed H.264 output, so you will get half broken frames.
To get this working, you would need to decode the raw H.264 frames back to raw yuv420p8.

Cu Selur

Ps.: moved the thread
Hello Selur,

   If this can help, it is possible pass the output of NVEnc to ffmepg, with this change

Code:
NVEnc_cmd = [NVEnc, '--avhw', '--input', source_path, '--lossless', '--output-format raw', '--output -', # output pipe
                 '|',
                 ffmpeg, '-i -', '-vcodec', 'rawvideo', '-pix_fmt', 'yuv420p',  '-f', 'rawvideo', '-'
                 ]


   But the out put is still the same

   Do you have any Idea ?

Thanks,
Dan
No clue.
I tried to dump the output of the 2 process.

The output of: "NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --output-format raw --output -" , is

 
Code:
Video
ID                          : 1
Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile              : High@L3
Format settings             : CABAC / 3 Ref Frames
Format settings, CABAC      : Yes
Format settings, Reference  : 3 frames
Codec ID                    : V_MPEG4/ISO/AVC
Duration                    : 2 min 2 s
Bit rate                    : 571 kb/s
Width                       : 720 pixels
Height                      : 300 pixels
Display aspect ratio        : 2.40:1
Frame rate mode             : Constant
Frame rate                  : 25.000 FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 0.106
Stream size                 : 8.30 MiB (100%)

having removed the option "--lossless" the output is much smaller.

The output of: "ffmpeg.exe -i E:\VideoTest\TestSubs\Test.mkv -vcodec h264 -pix_fmt yuv420p -f rawvideo -" , is

Code:
Video
ID                          : 1
Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile              : High@L3
Format settings             : CABAC / 4 Ref Frames
Format settings, CABAC      : Yes
Format settings, Reference  : 4 frames
Codec ID                    : V_MPEG4/ISO/AVC
Duration                    : 2 min 2 s
Bit rate                    : 373 kb/s
Width                       : 720 pixels
Height                      : 300 pixels
Display aspect ratio        : 2.40:1
Frame rate mode             : Constant
Frame rate                  : 25.000 FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 0.069
Stream size                 : 5.43 MiB (99%)
Writing library             : x264 core 161 r3027 4121277
Encoding settings           : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default                     : Yes
Forced                      : No
 
So it seems that the output is the same:

Codec ID                    : V_MPEG4/ISO/AVC
Frame rate                  : 25.000 FPS
Color space                : YUV
Chroma subsampling  : 4:2:0
Bit depth                    : 8 bits
Scan type                  : Progressive 

But is not working.
Using:
Code:
F:\Hybrid\64bit\NVEncC.exe --avhw --input "C:\Users\Selur\Desktop\TestSubs-1.mp4" --lossless --output-format raw --output - | F:\Hybrid\64bit\ffmpeg.exe -i - -an -vcodec rawvideo -pix_fmt yuv420p -f rawvideo - | f:\Hybrid\64bit\ffplay.exe -f rawvideo -pixel_format yuv420p -video_size 720x300 -i -
(not inside Vapoursynth, but directly in a terminal.)
I see the output, but there are also some outputs:
Code:
bitstream: Error writing file.
bitstream: Not enough disk space!

So the output seems to be correct (raw yvu420p), but there might be an issue with the reading in Python.

Quote:But is not working.
Top description. Sad

----
Yes,
Code:
NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --output-format raw --output -
and
Code:
ffmpeg.exe -i E:\VideoTest\TestSubs\Test.mkv -vcodec h264 -pix_fmt yuv420p -f rawvideo -
both create MPEG-4 AVC video.

In https://forum.doom9.org/showthread.php?p...ost1925640
Code:
ffmpeg  -i source_path -vcodec rawvideo -pix_fmt yuv420p -f rawvideo -
was used which creates raw yuv420p video.

Cu Selur

Ps.: you are aware that even if one gets this working, you won't be able to jump inside the clip etc. You can only go forward by one frame.
Willing to understand what is necessary to do, I think that we need to obtain with NVEnc a video with "uncompressed" frames.
This the first issue, because with NVEnc, i obtain the following output

Code:
NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output E:\VideoTest\TestSubs\Test.raw                    --------------------------------------------------------------------------------
E:\VideoTest\TestSubs\Test.raw
--------------------------------------------------------------------------------
NVEncC (x64) 7.31 (r2580) by rigaya, Aug 28 2023 13:07:12 (VC 1929/Win)
OS Version     Windows 10 x64 (19045) [UTF-8]
CPU            Intel Core i9-10900 @ 2.80GHz [TB: 4.82GHz] (10C/20T)
GPU            #0: NVIDIA GeForce RTX 3060 (3584 cores, 1837 MHz)[PCIe3x16][537.13]
NVENC / CUDA   NVENC API 12.1, CUDA 12.2, schedule mode: auto
Input Buffers  CUDA, 20 frames
Input Info     avcuvid: H.264/AVC, 720x300, 25/1 fps
AVSync         vfr
Vpp Filters    copyDtoD
Output Info    H.264/AVC Unknown @ Level auto
               720x300p 1:1 25.000fps (25/1fps)
Encoder Preset default
Rate Control   CQP  I:0  P:0  B:0 (lossless)
ChromaQPOffset cb:0  cr:0
Split Enc Mode auto
Lookahead      off
GOP length     250 frames
B frames       3 frames [ref mode: disabled]
Ref frames     3 frames, MultiRef L0:auto L1:auto
AQ             off
Others         mv:auto cabac deblock adapt-transform:auto bdirect:auto

encoded 3050 frames, 1034.95 fps, 9176.35 kbps, 133.46 MB
encode time 0:00:02, CPU: 3.1%, GPU: 15.0%, VE: 49.7%, VD: 27.0%, GPUClock: 1890MHz, VEClock: 1665MHz
frame type IDR   13
frame type I     13,  total size   1.01 MB
frame type P    769,  total size  40.79 MB
frame type B   2268,  total size  91.65 MB

The frames are compressed " H.264/AVC Unknown @ Level auto ", this fact is confirmed by Mediainfo

Code:
Video
ID                          : 1
Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile              : High 4:4:4 Predictive@L3
Format settings             : CABAC / 3 Ref Frames
Format settings, CABAC      : Yes
Format settings, Reference  : 3 frames
Codec ID                    : V_MPEG4/ISO/AVC
Duration                    : 2 min 2 s
Bit rate                    : 9 176 kb/s
Width                       : 720 pixels
Height                      : 300 pixels
Display aspect ratio        : 2.40:1
Frame rate mode             : Constant
Frame rate                  : 25.000 FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 1.699
Stream size                 : 133 MiB (100%)
Default                     : Yes
Forced                      : No

This is the first issue. I tried to fix the problem by using ffmpeg that is able to provide in output "true" raw video, but I think that the second pipe is "frozing" the process because I'm unable to forward the frames, while this is possible by using only ffmpeg, even if in this case the colors are wrong.

So the question is, it is possible to use NVEnc to produce in output "true" uncompressed frames ?

While using "ffmpeg" in pipe I obtaing the following output

Code:
NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output -| ffmpeg.exe -i - -vcodec rawvideo -pix_fmt yuv420p -loglevel debug -f rawvideo E:\VideoTest\TestSubs\Test2.raw
--------------------------------------------------------------------------------
-
--------------------------------------------------------------------------------
NVEncC (x64) 7.31 (r2580) by rigaya, Aug 28 2023 13:07:12 (VC 1929/Win)
OS Version     Windows 10 x64 (19045) [UTF-8]
CPU            Intel Core i9-10900 @ 2.80GHz [TB: 4.70GHz] (10C/20T)
GPU            #0: NVIDIA GeForce RTX 3060 (3584 cores, 1837 MHz)[PCIe3x16][537.13]
NVENC / CUDA   NVENC API 12.1, CUDA 12.2, schedule mode: auto
Input Buffers  CUDA, 20 frames
Input Info     avcuvid: H.264/AVC, 720x300, 25/1 fps
AVSync         vfr
Vpp Filters    copyDtoD
Output Info    H.264/AVC Unknown @ Level auto
               720x300p 1:1 25.000fps (25/1fps)
Encoder Preset default
Rate Control   CQP  I:0  P:0  B:0 (lossless)
ChromaQPOffset cb:0  cr:0
Split Enc Mode auto
Lookahead      off
GOP length     250 frames
B frames       3 frames [ref mode: disabled]
Ref frames     3 frames, MultiRef L0:auto L1:auto
AQ             off
Others         mv:auto cabac deblock adapt-transform:auto bdirect:auto

encoded 3050 frames, 767.88 fps, 9176.35 kbps, 133.46 MB
encode time 0:00:03, CPU: 1.4%, GPU: 8.6%, VE: 34.2%, VD: 20.6%, GPUClock: 1837MHz, VEClock: 1612MHz
frame type IDR   13
frame type I     13,  total size   1.01 MB
frame type P    769,  total size  40.79 MB
frame type B   2268,  total size  91.65 MB
ffmpeg version N-100400-g5f7dc836ce-gd76469378d+4 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
  configuration:  --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libaom --enable-libopenmpt --enable-version3 --enable-libsvthevc --enable-libsvtav1 --enable-libkvazaar --enable-libzmq --enable-libvmaf --enable-vapoursynth --disable-libdavs2 --enable-librav1e --enable-mbedtls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DZMQ_STATIC --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --extra-cflags=-DKVZ_STATIC_LIB
  libavutil      56. 62.100 / 56. 62.100
  libavcodec     58.115.102 / 58.115.102
  libavformat    58. 65.100 / 58. 65.100
  libavdevice    58. 11.103 / 58. 11.103
  libavfilter     7. 93.100 /  7. 93.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Splitting the commandline.
Reading option '-i' ... matched as input url with argument '-'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'rawvideo'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option 'E:\VideoTest\TestSubs\Test2.raw' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url -.
Successfully parsed a group of options.
Opening an input file: -.
[NULL @ 0000023ee2258500] Opening 'pipe:' for reading
[pipe @ 0000023ee2259600] Setting default whitelist 'crypto,data'
[AVIOContext @ 0000023ee2259800] Statistics: 1048576 bytes read, 0 seeks
pipe:: Invalid data found when processing input

So it seems that the pipe process is broken, and this could explain the frozen process.
Quote:So the question is, it is possible to use NVEnc to produce in output "true" uncompressed frames ?
No, NVEncC, can't output uncompressed frames.

Quote:NVEncC64.exe --avhw --input E:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output -| ffmpeg.exe -i - -vcodec rawvideo -pix_fmt yuv420p -loglevel debug -f rawvideo E:\VideoTest\TestSubs\Test2.raw
That does not work, since you output raw H.264, but tell ffmpeg that it reads raw uncompressed video with an unknown size.

Code:
NVEncC64.exe --avhw --inputE:\VideoTest\TestSubs\TestSubs-1.mp4 --lossless --output-format raw --output - | ffmpeg.exe -i - -an -vcodec rawvideo -pix_fmt yuv420p -f rawvideo Test_raw_yuv420p.raw
will output a raw yuv420 file.
Pages: 1 2 3 4