This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm whether you accept or reject these cookies being set.

A cookie will be stored in your browser regardless of choice to prevent you being asked this question again. You will be able to change your cookie settings at any time using the link in the footer.

[HELP] Crash when using Tensor RT with ESRGAN
#1
Hello,

I'm having an issue where the encoding crashes if I try to use Tensor RT when upscaling with ESRGAN. If I uncheck Tensor RT the encode works. It also works with Fusion. I'm using an RTX 4090 Strix and have iGPU disabled. More details below.

Build: Hybrid_dev_2023.01.01-162553

Steps I took
1. Load up Hybrid
2. Switch to Nvenc (no other settings)
3. Resize from 720x480 -> 1620x1080
4. Enable Vapoursynth ESRGAN resizer with Tensor RT, FP16 (crashes with or without this), and spline64
5. Encode

Any ideas what's going on or what I'm missing? Thanks for your help.

Debug log: https://drive.google.com/file/d/16_RNvE8...sp=sharing
Reply
#2
Will look at it after work and then report back.

Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
#3
Also check whether the Vapoursynth Preview works, or whether it shows an error.
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
#4
(21.02.2023, 08:34)Selur Wrote: Also check whether the Vapoursynth Preview works, or whether it shows an error.

Can confirm vs preview works. No errors generated.
Reply
#5
Script:
"F:\Hybrid-Temp\encodingTempSynthSkript_2023-02-20@22_02_00_5610.vpy"
### # Imports
import vapoursynth as vs
import os
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import sys
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
import site
# Adding torch dependencies to PATH
path = site.getsitepackages()[0]+'/torch_dependencies/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
path = path.replace('\\', '/')
os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
# source: 'F:\Gantz\disc1\gantz-disc1-001.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: top field first
# Loading F:\Gantz\disc1\gantz-disc1-001.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/Gantz/disc1/gantz-disc1-001.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2)
# setting field order to what QTGMC should assume (top field first)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True) # new fps: 29.97
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
clip = clip[::2]
from vsrealesrgan import RealESRGAN
# adjusting color space from YUV420P8 to RGBH for VsRealESRGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# resizing using RealESRGAN
clip = RealESRGAN(clip=clip, device_index=0, trt=True, trt_cache_path=r"F:\Hybrid-Temp") # 2880x1920
# resizing 2880x1920 to 1620x1080
# adjusting resizing
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, range_s="limited")
clip = core.fmtc.resample(clip=clip, w=1620, h=1080, kernel="spline64", interlaced=False, interlacedd=False)
# adjusting output color from: RGBS to YUV420P8 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 29.97fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Output
clip.set_output()
looks fine, encoding call:
"C:\Program Files\Hybrid\64bit\Vapoursynth\vspipe.exe" "F:\Hybrid-Temp\encodingTempSynthSkript_2023-02-20@22_02_00_5610.vpy" - -c y4m | "C:\Program Files\Hybrid\64bit\NVEncC.exe" --y4m -i - --fps 29.970 --codec h264 --profile high --level auto --sar 32:27 --lookahead 16 --vbr 0 --vbr-quality 0.00 --gop-len 0 --ref 3 --bframes 3 --direct auto --bref-mode disabled --no-b-adapt --mv-precision Q-pel --cabac --deblock --preset default --colorrange limited --colormatrix bt470bg --cuda-schedule sync --keyfile "F:\Hybrid-Temp\gantz-disc1-001-hybrid_2023-02-20@22_02_00_5610_02.qp" --output "F:\Hybrid-Temp\gantz-disc1-001-hybrid_2023-02-20@22_02_00_5610_03.264"
looks fine too.

The settings seem to work fine here.
You did close any Vapoursynth Previews, right?

=> can't reproduce the issue here.

Does it work if you now call the encoding call in a Windows command prompt?
According to the debug output the problem was that the Vapoursynth script failed.
My guess is that something else was using the gpu or was hindering the processing. (if you have a virus scanner running in parallel, make sure to set an exemption for the temp-folder)

Cu Selur
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
#6
try to run

"C:\Program Files\Hybrid\64bit\Vapoursynth\vspipe.exe" --info "F:\Hybrid-Temp\encodingTempSynthSkript_2023-02-20@22_02_00_5610.vpy"

and check if are shown strange messages
Reply
#7
@Whitehawkx: also send you a link to my current dev version and torch-addon (to be sure we use the same versions of ESRGAN&Co)
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply
#8
(21.02.2023, 19:53)Selur Wrote: @Whitehawkx: also send you a link to my current dev version and torch-addon (to be sure we use the same versions of ESRGAN&Co)

Thank you for the updated build and addon. This resolved my Tensor RT crash. I also added an exclusion for the temp folder just to be sure. FPS has drastically improved. Fantastic. Smile
Reply
#9
(22.02.2023, 03:15)Whitehawkx Wrote:
(21.02.2023, 19:53)Selur Wrote: @Whitehawkx: also send you a link to my current dev version and torch-addon (to be sure we use the same versions of ESRGAN&Co)

Thank you for the updated build and addon. This resolved my Tensor RT crash. I also added an exclusion for the temp folder just to be sure. FPS has drastically improved. Fantastic. Smile

Hi, i have the same problem, works ok with fusion, but with TENSOR RT CRASH TOO, please can send me the current addon, thanks in advance.

Greetings.
Reply
#10
send you updated links
----
Dev versions are in the 'experimental'-folder of my GoogleDrive, which is linked on the download page.
Reply


Forum Jump:


Users browsing this thread: 3 Guest(s)