NVEncC RTX SDR2HDR and SuperResolution

NVEncC RTX SDR2HDR and SuperResolution - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: NVEncC RTX SDR2HDR and SuperResolution (/thread-3753.html)

Pages: 1 2

NVEncC RTX SDR2HDR and SuperResolution - Dan64 - 30.06.2024

Hello Selur,

In NVEnc version 7.56 rigaya added

AI enhanced resize mode using NVIDIA VSR (Video Super Resolution)
AI enhanced SDR → HDR conversion by RTX Video SDK

since these AI option do not requires additional NVIDIA packages, are you planning to add support for them ?

Dan

RE: 4k (HDR) x265 → 2k (SDR) x264, slow speed and low resource utilization - Selur - 30.06.2024

Haven't decided yet, read the news, but haven't tested it. (I did test NVIDIA VSR, in the past, but wasn't impressed as it looked just like applying CAS.)

Cu Selur

Ps.: send you a link to a dev version, but for me at sdr2hdr does not work: https://github.com/rigaya/NVEnc/issues/550
PPs.: moved this to another thread, since it has nothing to do with the original thread.

RE: NVEncC RTX SDR2HDR and SuperResolution - Dan64 - 30.06.2024

I tested the AI Super Resolution and I was very impressed.

for testing I used the following clip: https://github.com/rigaya/NVEnc/assets/16306963/69ccef1c-adfe-4c37-91ef-91ff1d424de9

I compared the resized clip (1280x960) with: nvvfx, lanczos, spline64, nnedi3 (256 neurons)

And the NVDIA AI resizer is much better, here the comparison (using frame 1290): https://imgsli.com/Mjc1NTgz

It seems that the additional parameter "vsr-quality" is able to impact a little on final output quality.

Here a comparison using "vsr-quality=4": https://imgsli.com/Mjc1NTg1

Dan

RE: NVEncC RTX SDR2HDR and SuperResolution - Selur - 30.06.2024

Updated the nvenc dev build (same link), now includes vsr-quality control.
Ah, you are using the upscaler for cleaning your sources. I used more pristine sources. Wink

Cu Selur

RE: NVEncC RTX SDR2HDR and SuperResolution - Dan64 - 30.06.2024

I extended the comparison to the AI resizer available in Hybrid.

Here the comparison: https://imgsli.com/Mjc1NjMx

Based on this small test I assigned these (very subjective) quality scores

Score  Model

-----  ------------------------------------------------------------------------------

0:   AnimeSR_v2, RealCUGAN_se, RealCUGAN_pro                                        

5:   realsr-anime                                                

0:   AnimeSR_v1, RealCUGAN_no-se                                            

5:   RealESRGAN_4xplus_anime                                                

0:   GRLIR_Blind-image-SR                                                

0:   ngx-vsr_quality=4, AnimeScale_x2                                            

5:   AniScale2, Anime4KCPP                                            

0:   SwinIR-S_x2, BasicVSR++, Ani4Kv2_uc, Ani4Kv2_c, OpenProteus, AnimeJaNai_suc, 

       AnimeJaNai_uc, AnimeJaNai_c, SSIMD, DPID, RealSR, Waifu2x_nvk, Waifu2x

0:   GRLIR_Classical-image-SR_2x, ESRGAN_4x, ESRGAN_Plus_Multi_4x                                        

0:   FeMaSR_SRX2

Dan

RE: NVEncC RTX SDR2HDR and SuperResolution - Selur - 30.06.2024

You might want to try using BasicVSR++ (not the resizer) and then use one of the resizers.
(some of the models for VSGAN/VSMLRT might also be more suited)

Cu Selur

RE: NVEncC RTX SDR2HDR and SuperResolution - zspeciman - 30.06.2024

Are these for cartoons only or can also be used for real life footage? If it is the latter, a real life image comparison would be most welcome.
Human faces would be a good demo, not a closeup, but from a distance. On the right side a high resolution original image. Resize that image to 1/4 or less and upscale that image to full size going thru the various models you've listed and see which one matches the original the best. Just an idea.

RE: NVEncC RTX SDR2HDR and SuperResolution - Dan64 - 30.06.2024

(30.06.2024, 20:21)Selur Wrote: You might want to try using BasicVSR++ (not the resizer) and then use one of the resizers.
(some of the models for VSGAN/VSMLRT might also be more suited)

Cu Selur

I like BasicVSR++ to clean/restore clip, I'm not using it for resizing since there are better tools for this task.
I added it in the resizer comparison just because it was available.

Dan

RE: NVEncC RTX SDR2HDR and SuperResolution - Selur - 01.07.2024

Personally, I rarely use other upscalers than NNEDI3 (which doesn't really clean up).
I usually clean the source before upscaling.
(When using a machine learning upscaler, I usually use either RealESRGAN or FeMaSR.)

Quote:I added it in the resizer comparison just because it was available.

as a side note: DPID and SSIMD are downscalers, so when upscaling Hybrid will use the fallback scaler. Wink

Cu Selur

RE: NVEncC RTX SDR2HDR and SuperResolution - Dan64 - 01.07.2024

I'm not sure if "vsanimesr" is included in your last R68 addon package.
If not you should add it.
Even if "Tensor RT" and "Fusion" are not working, the filter is able to use the GPU and is still usable.

Do you know if HolyWu has intention to upgrade the filter ?

Dan

P.S.
Opened issue to HolyWu: https://github.com/HolyWu/vs-animesr/issues/5