NVEncC RTX SDR2HDR and SuperResolution

[Dan64]

Hello Selur,

   In NVEnc version 7.56 rigaya added

•      AI enhanced resize mode using NVIDIA VSR (Video Super Resolution)
  
•      AI enhanced SDR → HDR conversion by RTX Video SDK 
  

  since these AI option do not requires additional NVIDIA packages, are you planning to add support for them ?

Dan

[Selur]

Haven't decided yet, read the news, but haven't tested it. (I did test NVIDIA VSR, in the past, but wasn't impressed as it looked just like applying CAS.)

Cu Selur

Ps.: send you a link to a dev version, but for me at sdr2hdr does not work: https://github.com/rigaya/NVEnc/issues/550
PPs.: moved this to another thread, since it has nothing to do with the original thread.

[Dan64]

I tested the AI Super Resolution and I was very impressed.

for testing I used the following clip: https://github.com/rigaya/NVEnc/assets/1...ff1d424de9

I compared the resized clip (1280x960) with: nvvfx, lanczos, spline64, nnedi3 (256 neurons)

And the NVDIA AI resizer is much better, here the comparison (using frame 1290): https://imgsli.com/Mjc1NTgz

---

It seems that the additional parameter "vsr-quality" is able to impact a little on final output quality.

Here a comparison using  "vsr-quality=4": https://imgsli.com/Mjc1NTg1

Dan

[Selur]

Updated the nvenc dev build (same link), now includes vsr-quality control.
Ah, you are using the upscaler for cleaning your sources. I used more pristine sources.

Cu Selur

[Dan64]

I extended the comparison to the AI resizer available in Hybrid.

Here the comparison: https://imgsli.com/Mjc1NjMx

Based on this small test I assigned these (very subjective) quality scores

Score  Model
-----  ------------------------------------------------------------------------------
9.0:   AnimeSR_v2, RealCUGAN_se, RealCUGAN_pro                                        
8.5:   realsr-anime                                                
8.0:   AnimeSR_v1, RealCUGAN_no-se                                            
7.5:   RealESRGAN_4xplus_anime                                                
7.0:   GRLIR_Blind-image-SR                                                
6.0:   ngx-vsr_quality=4, AnimeScale_x2                                            
5.5:   AniScale2, Anime4KCPP                                            
5.0:   SwinIR-S_x2, BasicVSR++, Ani4Kv2_uc, Ani4Kv2_c, OpenProteus, AnimeJaNai_suc,
       AnimeJaNai_uc, AnimeJaNai_c, SSIMD, DPID, RealSR, Waifu2x_nvk, Waifu2x
4.0:   GRLIR_Classical-image-SR_2x, ESRGAN_4x, ESRGAN_Plus_Multi_4x                                        
3.0:   FeMaSR_SRX2

Dan

[Selur]

You might want to try using BasicVSR++ (not the resizer) and then use one of the resizers.
(some of the models for VSGAN/VSMLRT might also be more suited)

Cu Selur

[zspeciman]

Are these for cartoons only or can also be used for real life footage?  If it is the latter, a real life image comparison would be most welcome.
Human faces would be a good demo, not a closeup, but from a distance. On the right side a high resolution original image. Resize that image to 1/4 or less and upscale that image to full size going thru the various models you've listed and see which one matches the original the best. Just an idea.

[Dan64]

You might want to try using BasicVSR++ (not the resizer) and then use one of the resizers.
(some of the models for VSGAN/VSMLRT might also be more suited)

Cu Selur

I like BasicVSR++  to clean/restore clip,  I'm not using it for resizing since there are better tools for this task.
I added it in the resizer comparison just because it was available.

Dan

[Selur]

Personally, I rarely use other upscalers than NNEDI3 (which doesn't really clean up).
I usually clean the source before upscaling.
(When using a machine learning upscaler, I usually use either RealESRGAN or FeMaSR.)

I added it in the resizer comparison just because it was available.

as a side note: DPID and SSIMD are downscalers, so when upscaling Hybrid will use the fallback scaler.

Cu Selur

[Dan64]

I'm not sure if "vsanimesr" is included in your last R68 addon package. 
If not you should add it.
Even if "Tensor RT" and "Fusion" are not working, the filter is able to use the GPU and is still usable.

Do you know if HolyWu has intention to upgrade the filter ?

Dan

P.S.
Opened issue to HolyWu: https://github.com/HolyWu/vs-animesr/issues/5