Selur's Little Message Board
[HELP] Crash when using Tensor RT with ESRGAN - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: [HELP] Crash when using Tensor RT with ESRGAN (/thread-3095.html)

Pages: 1 2


Crash when using Tensor RT with ESRGAN - Whitehawkx - 21.02.2023

Hello,

I'm having an issue where the encoding crashes if I try to use Tensor RT when upscaling with ESRGAN. If I uncheck Tensor RT the encode works. It also works with Fusion. I'm using an RTX 4090 Strix and have iGPU disabled. More details below.

Build: Hybrid_dev_2023.01.01-162553

Steps I took
1. Load up Hybrid
2. Switch to Nvenc (no other settings)
3. Resize from 720x480 -> 1620x1080
4. Enable Vapoursynth ESRGAN resizer with Tensor RT, FP16 (crashes with or without this), and spline64
5. Encode

Any ideas what's going on or what I'm missing? Thanks for your help.

Debug log: https://drive.google.com/file/d/16_RNvE8pIivBhqWnRaFtq5kR4PwGtRyh/view?usp=sharing


RE: Crash when using Tensor RT with ESRGAN - Selur - 21.02.2023

Will look at it after work and then report back.

Cu Selur


RE: Crash when using Tensor RT with ESRGAN - Selur - 21.02.2023

Also check whether the Vapoursynth Preview works, or whether it shows an error.


RE: Crash when using Tensor RT with ESRGAN - Whitehawkx - 21.02.2023

(21.02.2023, 08:34)Selur Wrote: Also check whether the Vapoursynth Preview works, or whether it shows an error.

Can confirm vs preview works. No errors generated.


RE: Crash when using Tensor RT with ESRGAN - Selur - 21.02.2023

Script:
"F:\Hybrid-Temp\encodingTempSynthSkript_2023-02-20@22_02_00_5610.vpy"
### # Imports
import vapoursynth as vs
import os
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import sys
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
import site
# Adding torch dependencies to PATH
path = site.getsitepackages()[0]+'/torch_dependencies/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
path = path.replace('\\', '/')
os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
# source: 'F:\Gantz\disc1\gantz-disc1-001.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: top field first
# Loading F:\Gantz\disc1\gantz-disc1-001.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/Gantz/disc1/gantz-disc1-001.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2)
# setting field order to what QTGMC should assume (top field first)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True) # new fps: 29.97
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
clip = clip[::2]
from vsrealesrgan import RealESRGAN
# adjusting color space from YUV420P8 to RGBH for VsRealESRGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# resizing using RealESRGAN
clip = RealESRGAN(clip=clip, device_index=0, trt=True, trt_cache_path=r"F:\Hybrid-Temp") # 2880x1920
# resizing 2880x1920 to 1620x1080
# adjusting resizing
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, range_s="limited")
clip = core.fmtc.resample(clip=clip, w=1620, h=1080, kernel="spline64", interlaced=False, interlacedd=False)
# adjusting output color from: RGBS to YUV420P8 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 29.97fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Output
clip.set_output()
looks fine, encoding call:
"C:\Program Files\Hybrid\64bit\Vapoursynth\vspipe.exe" "F:\Hybrid-Temp\encodingTempSynthSkript_2023-02-20@22_02_00_5610.vpy" - -c y4m | "C:\Program Files\Hybrid\64bit\NVEncC.exe" --y4m -i - --fps 29.970 --codec h264 --profile high --level auto --sar 32:27 --lookahead 16 --vbr 0 --vbr-quality 0.00 --gop-len 0 --ref 3 --bframes 3 --direct auto --bref-mode disabled --no-b-adapt --mv-precision Q-pel --cabac --deblock --preset default --colorrange limited --colormatrix bt470bg --cuda-schedule sync --keyfile "F:\Hybrid-Temp\gantz-disc1-001-hybrid_2023-02-20@22_02_00_5610_02.qp" --output "F:\Hybrid-Temp\gantz-disc1-001-hybrid_2023-02-20@22_02_00_5610_03.264"
looks fine too.

The settings seem to work fine here.
You did close any Vapoursynth Previews, right?

=> can't reproduce the issue here.

Does it work if you now call the encoding call in a Windows command prompt?
According to the debug output the problem was that the Vapoursynth script failed.
My guess is that something else was using the gpu or was hindering the processing. (if you have a virus scanner running in parallel, make sure to set an exemption for the temp-folder)

Cu Selur


RE: Crash when using Tensor RT with ESRGAN - Dan64 - 21.02.2023

try to run

"C:\Program Files\Hybrid\64bit\Vapoursynth\vspipe.exe" --info "F:\Hybrid-Temp\encodingTempSynthSkript_2023-02-20@22_02_00_5610.vpy"

and check if are shown strange messages


RE: Crash when using Tensor RT with ESRGAN - Selur - 21.02.2023

@Whitehawkx: also send you a link to my current dev version and torch-addon (to be sure we use the same versions of ESRGAN&Co)


RE: Crash when using Tensor RT with ESRGAN - Whitehawkx - 22.02.2023

(21.02.2023, 19:53)Selur Wrote: @Whitehawkx: also send you a link to my current dev version and torch-addon (to be sure we use the same versions of ESRGAN&Co)

Thank you for the updated build and addon. This resolved my Tensor RT crash. I also added an exclusion for the temp folder just to be sure. FPS has drastically improved. Fantastic. Smile


RE: Crash when using Tensor RT with ESRGAN - sowbran - 22.02.2023

(22.02.2023, 03:15)Whitehawkx Wrote:
(21.02.2023, 19:53)Selur Wrote: @Whitehawkx: also send you a link to my current dev version and torch-addon (to be sure we use the same versions of ESRGAN&Co)

Thank you for the updated build and addon. This resolved my Tensor RT crash. I also added an exclusion for the temp folder just to be sure. FPS has drastically improved. Fantastic. Smile

Hi, i have the same problem, works ok with fusion, but with TENSOR RT CRASH TOO, please can send me the current addon, thanks in advance.

Greetings.


RE: Crash when using Tensor RT with ESRGAN - Selur - 22.02.2023

send you updated links