2D Cleaners and Rescale!!

[Cauptain]

Hello Selur, how are you? Have you already been vaccinated? I hope you are well.

I'm trying to resize and encode 720p video with the best quality possible with 2D Cleaner filters, unfortunately all I tried didn't work so well.

Tried:

Santiag
AwarpSharp2
Santiag
Nnedi3

All leave grids in the image.



I uploaded 1 minute of the video to help and was wondering what the best filters would be and the order to get the best quality when putting in 1080p.

1 minute Video

Don't care about file size, but I'd like to at least leave it clean, like it's original.

Thanks,


Claudio

[Selur]

Okay,... I don't think any normal filter can clean this up properly.
That said using

# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
from vsgan import VSGAN

# Import scripts
import adjust

clip = core.lsmas.LibavSMASHSource(source="C:/Users/Selur/Desktop/Hora.de.Aventura-00.03.05.366-00.04.05.766.mp4")
# making sure input color matrix is set as 709
clip = core.resize.Point(clip, matrix_in_s="709",range_s="limited")
# making sure frame rate is set to 30
clip = core.std.AssumeFPS(clip=clip, fpsnum=30, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

# needed since otherwise vsgan will crash on my GPU due to VRAM shortage
clip = core.fmtc.resample(clip=clip, kernel="spline16", w=720, h=406, interlaced=False, interlacedd=False)

# adjusting color space from YUV420P8 to RGB24 for vsLevels
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="limited")

# creating a VSGAN instance
vsgan = VSGAN("cuda")
# setting model
model = "J:/VapourSynth64Portable_VSGAN/vsgan_models/BSRGAN.pth"
vsgan.load_model(model)
# running vsgan with the selected model
clip = vsgan.run(clip=clip)

# convert to YUV420P10 for encoding
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="709", range_s="limited")

clip.set_output()

Produces really nice output:

Problem is:
a. BSRGAN increases width&height by a factor of 4 (so if you want a lower resolution additional resizing your be needed
b. I had to reduce the resolution to now run out of VRAM and at least on my Geforce GTX 1070ti, I get ~0.23fps for this (cpu isn't really used) Encoding the 1822 frames is estimated to take ~2hours.

There probably is a way to get this done without using BSRGAN (https://github.com/cszn/BSRGAN) or the way I use it through vsgan, my cleverly creating some mask and similar, but I don't know a simple filter I could add for Hybrid that would directly help with this.


Cu Selur

[Selur]

You can get quite decent results with "Vapoursynth->Frames->Resize->Resizer->Waifu2x nvk" with 'Model ''upconv_7_anime_syslte_art_rgb' and 'Noise' set th 'highest'.
BSRGAN is still quite a bit better (in my opinion), but a lot slower.

I converted the first 1465 frames mit BSRGAN (then I accidentally aborted the encode), see:
https://drive.google.com/file/d/1rhDwx5R...sp=sharing

Cu Selur

[Cauptain]

I was afraid you would tell me that. I was never able to correctly configure ESGRAN.

I did tests with Video2x and Waifu2x-Extension-Gui and both took several and several minutes to convert 1 minute of video.

Even with a GTX 2080TI it wasn't what I expected.

I also tested Topaz Video AI and despite being much faster the image is not so good.

I would just need a way then that would make it less bad, but only using the Hybrid filters, similar to what I used to do in Avisynth.

Is it possible to put Scanline by Hybrid? When I use the shader in MPC it is very good for watching.

Anyway thanks for the tip about BSRGAN, I'll try to read a little more about it and see if I can install it.

Thank you very much Selur.



Claudio

---

You can get quite decent results with "Vapoursynth->Frames->Resize->Resizer->Waifu2x nvk" with 'Model ''upconv_7_anime_syslte_art_rgb' and 'Noise' set th 'highest'.
BSRGAN is still quite a bit better (in my opinion), but a lot slower.

I converted the first 1465 frames mit BSRGAN (then I accidentally aborted the encode), see:
https://drive.google.com/file/d/1rhDwx5R...sp=sharing

Cu Selur

Wow, watch the video converted and its perfect for me.

OMG, perfect because only Hybrid used. Try that config now!!

Many thanks Selur. 


Claudio

[Selur]

I explained in https://forum.selur.net/Thread-Why-VSGAN...-supported how to setup a portable vegan, but using Waifu in Hybrid ist probably easier.

Cu Selur

[Selur]

I uploaded a version of the clip which was created with "Vapoursynth->Frames->Resize->Resizer->Waifu2x nvk" with 'Model ''upconv_7_anime_syslte_art_rgb' and 'Noise' set to 'highest':
https://drive.google.com/file/d/1O6yeA2F...sp=sharing
creation was six times faster than BSRGAN on my Geforce GTX 1070ti. (~1.5fps)

Cu Selur

Ps.: since I usually don't filter so aggressively I also created a post over at videohelp to see if may be someone else has a nice idea. (may be some filter I could add to Hybrid that would work faster with a similar result )

[Selur]

Btw. I adjusted Hybrid to use VSGAN after installing a ~6.67GB (download size; 11.1GB install size) 'addon'.
The size of the 'addon' is so huge since it includes a 5.5 GB folder with models for VSGAN.
Assuming you have an NVIDIA card I can send you a link to a dev version with this feature and a link to the 'addon'.

Cu Selur

[sowbran]

Btw. I adjusted Hybrid to use VSGAN after installing a ~6.67GB (download size; 11.1GB install size) 'addon'.
The size of the 'addon' is so huge since it includes a 5.5 GB folder with models for VSGAN.
Assuming you have an NVIDIA card I can send you a link to a dev version with this feature and a link to the 'addon'.

Cu Selur

Hi, i like to try the version of Hybrid with VSGAN, i follow the tutorial with Vapoursynth portable, but Vapoursynth gets freeze when i check the script in my pc (Nvidia 3060), i dont know why it falls, because in my notebook works (Nvidea 1650 ti). For that reason i like to try this version of Hybrid. I hope you can share with my this dev version. Thank you for all your work, you´re amazing. Greeetings

[Selur]

Send you the links via pm.

Cu Selur

[Selur]

Ha! Today came a new filter out DPIR (https://github.com/HolyWu/vs-dpir) which is really nice for that content and a lot faster than BSGRAN and Waifu2x


Cu Selur