Improving CodeFormer videos

[Dan64]

Please note that usually you add in all the scripts using CUDA, the following code:

# Adding torch dependencies to PATH
path = site.getsitepackages()[0]+'/torch_dependencies/bin/'
ctypes.windll.kernel32.SetDllDirectoryW(path)
path = path.replace('\\', '/')
os.environ["PATH"] = path + os.pathsep + os.environ["PATH"]

In the case of CodeFormer you don't add this code, because "dlib" is able to find the CUDA itself, but If you want help "dlib" to find CUDA on PC with a different installation you should add this code.

Please add it and let me know if it works.

Dan

[Selur]

Bingo! Adding torch dependencies to path does help!
(Hybrid didn't add it since it wasn't necessary before.)

Cu Selur

[Dan64]

With CUDA enabled "dlib", CodeFormer it is a perfect match for RealESRGAN.

See "28 Days Later..."

https://imgsli.com/MjQwMzE2

While with "retinaface" CodeFormer is not usable to encode videos (*) because are introduced too many artifacts

Dan

(*) With pictures "retinaface" is Ok

[Selur]

That said, happy that it's a good solution for your usage.
https://imgsli.com/MjQwMzE2
Personally I think that looks horrible, face enhancement is way to detailed for the rest and the background looks too blocky.
Maybe try FeMaSR instead, might look good or really horrible depending on the source.

Cu Selur

Ps.: did the version I send you also support FeMaSR for CodeFormer (I added it, but don't know whether the version I send you already supported it)
PPs.: send you a link to my current dev.

[Dan64]

Hello Selur,

  I tested your dev version and I found the following problems:

   1) FeMaSR 1.2 TensorRT is not working (actually never worked in RT on my PC since the bump version to 1.2)
       I had to downgrade to  FeMaSR 1.1 TensorRT to be able to use the filter in RT mode
   2) The FeMaSR included in CodeFormer is using fp32 and is too slow to be practically used, please use as default fp16
   3) In the FeMaSR included in CodeFormer is not used the parameter trt_min_subgraph_size, I suggest to set it = 5 (should be good for every one)

  I do agree with you that FeMaSR could be better than RealESRGAN, see the comparison:

  https://imgsli.com/MjQwMzY2

  The problem is that is too slow, on my PC the speed is about 3fps (with TensoRT fp16) while with RealESRGAN I can get a speed around 6fps.
  I need to do more tests to understand better when to use one of them.

Thanks
Dan

[Selur]

regarding 1: works fine here
regarding 2&3: I'll adjust Hybrid to use FP16 and trt_min_subgraph_size=5
Regarding speed: Yes, FeMaSR is slower.

Cu Selur

[Dan64]

regarding 1: works fine here

Probably my RTX3060 is not big enough to support the Tensor-RT implementation of version 1.2

[Dan64]

regarding 2&3: I'll adjust Hybrid to use FP16 and trt_min_subgraph_size=5

In the last dev version is generate: trt_min_subgraph_size="5"  (5 as string instead of an integer)

[Selur]

argh,.. will fix

[Selur]

Send you a new link.

Cu Selur