Questions regarding BasicVSR++

[Dan64]

I noted that in Hybrid BasicVSR++ is available under 2 different filter tab:

1) Vapoursynth->Frame->Resize

   where the available models are: Vimeo-90K (BI), Vimeo-90K (BD)
   It is not clear to me the difference of these 2 models.

2) Vapoursynth->Other

    where the available models are: NITRE 2021 (1), NITRE 2021 (2), NITRE 2021 (3)
    Even in this case it is not clear to me difference of these 3 models.

Could you explain the usage of these 2 different BasicVSR++ modules.

At the address https://github.com/open-mmlab/mmediting/.../README.md

there is a reference on these models, but nothing that helps in understanding the difference, moreover it seems that all these models are used for the Video Super-Resolution.

[Selur]

Difference between the models is just that they are trained with different content and thus will have different effects on different content.
NTIRE = New Trends in Image Restoration and Enhancement workshop and challenges on image and video processing. These are the models that were used in last years NTIRE challenges. See: https://data.vision.ee.ethz.ch/cvl/ntire21/

regarding 1.:
These models are used for upscaling.

regarding 2:
Models are used for denoising and restoration.

Cu Selur

[Dan64]

I performed some tests, for resize BasicVSR++ is too slow to be used in practice.

For resize the best combination is to use NNEDI3 for resize and BasicVSR++ (with model: NTIRE 2021 (3)) to improve the image quality.

In this case the speed is about 5 fps (with RTX3060 and FP16 enabled) and the final quality comparable with RealESERGAN.

[Selur]

ML stuff is mostly slow, espescially when using higher resolution.

For resize the best combination is to use NNEDI3 for resize and BasicVSR++ (with model: NTIRE 2021 (3)) to improve the image quality.

Happy you found a meaningful way to use it.

Cu Selur

[protectron]

I'm also running  BasicVSR++(filter) on SD content on a RTX3060 and getting about 4-5 fps.

Task Manager reports my GPU usage is only at ~5% however and CPU ~25%.

[Selur]

Sadly, vs-basicbrpp doesn't really offer any options for tuning aside from tiles, which usually only help with the vram usage. gpu usage seems low, I assume you did try enabling fp16?

[protectron]

I assume you did try enabling fp16?

Yes, it's more or less the same. My fans run really loud though like it's working harder than 5% so I'm thinking it may be some sort of reporting error.

[Selur]

I just checked on my 4080 and I get way hight GPU usage and ~14+ fps.


Cu Selur

[protectron]

Here's what is typical on my machine

[Selur]

Don't really know why it is the way it is on your system, but ...
What source filter do you use in your script? Does changing the source filter have an impact on this for you?
(when using machine learning stuff, I often use software - not hardware - based source filters)
Does it change if you set 'Limit thread count' to 1?

Cu Selur