Request: Improving RealSR quality

[Dan64]

Recently I found the following spin-off of the project RealSR: https://github.com/xinntao/Real-ESRGAN

The project is called Real-ESRGAN, and there is not a specific Vapoursynth filter, but the models are "compatible" with the models used by RealSR.

The models are included in the distribution: https://github.com/xinntao/Real-ESRGAN/r...indows.zip

And are the following:

realesrgan-x4plus (for real-life video upscaling)
realesrgan-x4plus-anime (for anime video upscaling)
realesrnet-x4plus (model trained on DIV2K dataset)

the realesrgan-x4plus is probably the best model for upscaling videos (at cost of speed).

To test the model i renamed "realesrgan-x4plus" in "x4" and then I override the files in "Hybrid\64bit\vsfilters\ResizeFilter\RealSR\models\models-DF2K_JPEG".  Given the compatibility with RealSR this was the easiest solution to use hybrid to test the model.

To test the model I downsized a short movie (20s) to the resolution of 640x346 (ratio 1.85:1) and then I resized it to 1920x1040.

This is the image resized with Lanczos4



This is image is very near to the source (fake Blu-Ray) since I used for testing a clip of the movie "28 Days Later" that was filmed using a Canon XL-1s standard-definition camcorder at 720x576 resolution and probably the Blu-Ray was obtain by up-scaling with Lanczos (or something similar) the original movie.

The following picture was obtained using RealSR included in Hybrid



And finally this is the picture obtained using realesrgan-x4plus



As it is possible to see it is not necessary to perform 4x zoom to observe the details, the difference is very clear and the up-scaled image with realesrgan-x4plus contains more details than the original movie.

REQUEST

It would be nice if Hybrid will include in RealSR filters also the models described above:

• realesrgan-x4plus
  
• realesrgan-x4plus-anime
  
• realesrnet-x4plus
  

Thanks

[Selur]

Hybrid does support https://github.com/HolyWu/vs-realesrgan/ which is a Vapoursynth port of that project in it's torch addon (before I named it VSGAN addon).
-> nothing to do
I also posted some info how I build the addon, see: https://forum.selur.net/thread-1858-post-13770.html, I just don't offer the addon as a general download since it's 10GB atm. and having tons of bots&co download that would kill the small (and cheap webserver the forum and the main page are hosted).

Cu Selur

[Dan64]

Hello Selur !

   The RealESRGAN resizer available in the last dev-version was exactly what I was looking for. You are always one step ahead!

   The quality of output is almost the same (maybe even better) of the one obtained by using realesrgan-x4plus model with the RealSR filter. But the RealESRGAN resizer is about twice faster (0.80fps vs 0.40fps). But even better the RealESRGAN resizer has the 2X model (scale=2), and using this model the speed increase to 3.14fps.

   I have always complained that good AI resizers are too slow to be used for upscaling long videos, but this is not more the case with the Real ESRGAN resizer.

Many thanks Selur!

[Dan64]

Hello Selur!

  just for comparison this is the same frame encoded using RealESRGAN_2X



  the quality is in between the one obtained using RealESRGAN_4X (the best) and RealSR_4X but RealESRGAN_2X is about 4 time faster than RealESRGAN_4X and the small decrease in quality is acceptable.

Thanks again!

[Selur]

Happy RealESRGAN work for you.
You might also want to test some of the other resizers.

Cu Selur

[Dan64]

Happy RealESRGAN work for you.
You might also want to test some of the other resizers.

Cu Selur

I tested the other resizers. If the quality of the source is not too bad, I found that "Waifu2x nvk" is a good resizer, not too slow to be used as quality resizer.
It is not able to perform "miracles" but if the quality of the source is good enough the "miracles" are not necessary.

Here some results

[Selur]

Also try whether adding CAS (contrast adaptive sharpening), after the resizing does help.

Cu Selur

[Dan64]

Also try whether adding CAS (contrast adaptive sharpening), after the resizing does help.

Cu Selur

Wow adding CAS add a very nice extra sharpening, here some results.


-- by Selur: removed image links, since Malwarebytes keeps poping up that it blocked the ip due to possible trojaners ---

[Selur]

Happy that helped a bit.

Cu Selur