Problems in using RealESRGAN in Hybrid 2022.11.28.1

[Dan64]

I installed all the packages described in your guide: [INFO] Portable Vapoursynth (Windows) with pytorch&co

But when I try to use RealESRGAN I get the following error:

Failed to evaluate the script:
Python exception: DLL load failed while importing tensorrt: Impossibile trovare la procedura specificata.

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2819, in vapoursynth._vpy_evaluate
File "src\cython\vapoursynth.pyx", line 2820, in vapoursynth._vpy_evaluate
File "E:\MEGAsync Downloads\20.000 leghe sotto i mari (2004)\tempPreviewVapoursynthFile18_53_42_947.vpy", line 27, in
from vsrealesrgan import RealESRGAN
File "D:\Programs\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsrealesrgan\__init__.py", line 9, in
import tensorrt
File "D:\Programs\Hybrid\64bit\Vapoursynth\Lib\site-packages\tensorrt\__init__.py", line 135, in
from .tensorrt import *
ImportError: DLL load failed while importing tensorrt: The specified procedure could not be found.

---

In the case it could be useful.

In my previous install (that was working) I was using vsrealesrgan-3.0.0 that is using "onnx" files, while the version currently installed vsrealesrgan-4.0.1 is using "pth" files.

[Selur]

Yes, a lot changed in RIFE 4.0 not only the dependencies also the interface.
-> I'll package my current dev and will send you a link to it and my current torchAddon&Co via PM in ~1/2hr.

Cu Selur

[Selur]

Send you a pm.

Cu Selur

[Dan64]

Hello Selur!

   I downloaded your dev version, the torch addons, and now RealESRGAN in Hybrid is working as expected.

   I done some tests and despite the jump in version (from 3.0.0 to 4.0.1) I was unable to observe significant improvement in quality for RealESRGAN.
  
   The only big improvement was on the start-up time when the TensorRT is enabled. Previously it was necessary to wait till 40min before the encoding could start and was generated a cache file with name like: "TensorrtExecutionProvider_TRTKernel_graph_torch-jit-export_18011103394555784438_1_0_fp16.engine".
   Now the start-up with TensorRT is less than 1min and are not generated the cache files (the cache is in memory ?).

    It seems that this improvement in using TensorRT has impacted on the encoding speed. Now the encoding using "x4plus_anime_6B" (old name) is 15% slower and the encoding using animevideo_v3 is 27% slower respect to animevideo_v2 with no observable difference in quality.

    I don't know if this decrease in speed is due to the fact that the version 4.0.1 dropped the support to ONNX, there is a way to enable it ? i.e. convert the "pth" models in "onnx" ?

    I noted also that you included RealCUGAN. I tested all the 3 models, but on my sample only the "se" model was able to provide a good quality, comparable with RealESRGAN animevideo.
    It seems that it is missing the support for TensorRT in RealCUGAN and its encoding speed is slower respect to RealESRGAN animevideo when TensorRT is enabled (4.06fps vs 13.58fps).

I Thank you very much for the support.
Dan

[Selur]

Regarding RealESRGAN speed:
Can't really say much about the speed of RealESRGAN, since I don't really use it much and thus I haven't noticed a change.

there is a way to enable it ?

No, vs-realesrgan only supports pytorch instead of ONNX now. You would have to contact HolyWu over in git to know why this change happened.
(https://github.com/HolyWu/vs-realesrgan)

Regarding RealCUGAN:
Depends on the source, for some RealESRGAN models seem nicer for others RealCUGAN.
For the lack of TensorRT support complain to: https://github.com/Kiyamou/VapourSynth-R...cnn-Vulkan

Since RealCUGAN and RealESRGAN are quite different, it's not surprising that their speed is different.

Cu Selur

[Dan64]

Hello Selur!

   I opened a discussion on this topic: v4.0.1 discussions

Thanks,
Dan

[Dan64]

Hello Selur

    looking to the benchmark provided by HolyWu, it seems that by setting "num_streams=2" as in the example below:

    clip = RealESRGAN(clip=clip, model=3, device_index=0, num_streams=2, trt_cache_path="C:/Temp", trt_max_workspace_size=1073741824)

   there is an increase in speed between 5% and 40%, could you add this parameter in Hybrid ?

Thanks,
Dan

[Selur]

Sure, no problem, missed that parameter when adjusting to the latest changes.

Cu Selur

[Dan64]

Hello Selur,

   another parameter missed is "trt" that is "False" by default. That's the reason why I observed lower speed respect to the 3.0.0. Becasuse even if in Hybrid is specified to use "Tensor RT", the flag "trt=True" is not set in the generated script and so the Tensor is not used. Please add it.

Thanks,
Dan

[Selur]

Hmm,.. somehow RealESRGAN seem broken when using trt.
when using

clip = RealESRGAN(clip=clip, device_index=0, trt=True, trt_cache_path="G:\Temp", num_streams=1) # 5120x2880

I get:

Python exception: Ran out of input

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2866, in vapoursynth._vpy_evaluate
File "src\cython\vapoursynth.pyx", line 2867, in vapoursynth._vpy_evaluate
File "G:\Temp\tempPreviewVapoursynthFile17_11_27_438.vpy", line 32, in
clip = RealESRGAN(clip=clip, device_index=0, trt=True, trt_cache_path="G:\Temp", num_streams=1) # 5120x2880
File "I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsrealesrgan\__init__.py", line 284, in RealESRGAN
module = [torch.load(trt_engine_path) for _ in range(num_streams)]
File "I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\vsrealesrgan\__init__.py", line 284, in
module = [torch.load(trt_engine_path) for _ in range(num_streams)]
File "I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\serialization.py", line 795, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "I:\Hybrid\64bit\Vapoursynth\Lib\site-packages\torch\serialization.py", line 1002, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
EOFError: Ran out of input

(same happens with different source, SD and HD)
Whether I use "G:/Temp" or "G:\Temp" does not seem to change that.
Posted it to github (https://github.com/HolyWu/vs-realesrgan/issues/18)