Cleaning old 640x480 AVI clips

[Al74]

Dear Selur and the forum Members,

Once upon a time Selur was very kind to clean my AVI clip, which contained typical problems present in many similar clips I had. I would be grateful for some help on how to clean it (which filters to use and so on). I tried different options but I am not satisfied.

Below I attach two files, original one and cleaned one.

Thank you in advance.

https://we.tl/t-gIk2Q28K3b

[Selur]

You might want to share:
What you tried.
What it is you are aiming for. Just cleaning? Are you aiming to upscale this to xy?
Are you aiming to use Avisynth or Vapoursynth or none of those?

Is using ai based filters an option (requires additional addon + modern NVIDIA card)?
(ai based upscaling often includes denoising&co)

Is waiting xy minutes per frame an option?

Something like FeMaSR can do some nice stuff, but is even slow on high-end NVIDIA graphics cards, especially when aiming to resize for a resize factor above 2.


Not using ai, for dehalo and deringing, something like:

# applying dehalo using YAHR
clip = havsfunc.YAHR(clip, blur=1, depth=16)
clip = havsfunc.FineDehalo(clip)
# adjusting color space from YUV422P8 to YUV444P16 for vsGLSLAntiRing
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, range_s="full")
# applying anti-aliasing using glsl filter
with open("i:/Hybrid/64bit/vsfilters/GLSL/parameterized/antiring.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('const float strength = 1.0;', 'const float strength = 1;')
glsl = glsl.replace('const float radius = 0.5;', 'const float radius = 0.5;')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)

seems okay.

Cu Selur

[Al74]

Dear Selur and the forum Members,

Once upon a time Selur was very kind to clean my AVI clip, which contained typical problems present in many similar clips I had. I would be grateful for some help on how to clean it (which filters to use and so on). I tried different options but I am not satisfied.

Below I attach two files, original one and cleaned one.

Thank you in advance.

https://we.tl/t-gIk2Q28K3b

You might want to share:
What you tried.
What it is you are aiming for. Just cleaning? Are you aiming to upscale this to xy?
Are you aiming to use Avisynth or Vapoursynth or none of those?

Is using ai based filters an option (requires additional addon + modern NVIDIA card)?
(ai based upscaling often includes denoising&co)

Is waiting xy minutes per frame an option?

Something like FeMaSR can do some nice stuff, but is even slow on high-end NVIDIA graphics cards, especially when aiming to resize for a resize factor above 2.


Not using ai, for dehalo and deringing, something like:

# applying dehalo using YAHR
clip = havsfunc.YAHR(clip, blur=1, depth=16)
clip = havsfunc.FineDehalo(clip)
# adjusting color space from YUV422P8 to YUV444P16 for vsGLSLAntiRing
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, range_s="full")
# applying anti-aliasing using glsl filter
with open("i:/Hybrid/64bit/vsfilters/GLSL/parameterized/antiring.glsl") as glslf:
  glsl = glslf.read()
glsl = glsl.replace('const float strength = 1.0;', 'const float strength = 1;')
glsl = glsl.replace('const float radius = 0.5;', 'const float radius = 0.5;')
clip = core.placebo.Shader(clip=clip, shader_s=glsl, width=clip.width, height=clip.height)

seems okay.

Cu Selur

Selur,
Thank you for the reply. Let me give you some background.
My goal is to clean my AVI files and then upscale them 2x or even 4x if one can see clear diiference/advantage. As you have written, one can use either AI-based upscalers or more traditional ones. Due to constant progress in the field, it may be done later on...
I have old GTX 1060 6GB GPU so it can be chalange for some AI models to get job done. BUT if the results are very good, I could upscale some clips from different places to finish some presentations (mix of photos and short clips).
So for the time being, the most important thing for me is to clean these files for further future editing.
Coming back to my attempts:
I have also tried YAHR , Blur 1, but used Depth 32, not 16 as you proposed.
YAHR is effective to remove halo BUT also seems to affect other parts of clip, it removes some fine detail a little bit.
After applying YAHR 1, I added TemporalDegran and it helped me to remove "dancing artifacts" but also influenced the detail.
I have also used denoise FFT3D filter but as I can see one should experiment with different Sigma value, I must explore it more.
So you apart from YAHR 1 you also added FineDehalo, could you write why you used it apart from YAHR 1, to strengthen the dehalo effect even more?
What about any additional denoisng etc?
The file you created sometime ago shows much less noise then procedure you showed above, I just reproduced it.
Is the picture comparison done using original and filtered according to above script only ?
PS As far as upscaling is concerned, of course I could also be interested in using Avisynth or Vapoursynth methods. Sometimes I use NNEDI3 method.

[Selur]

NNEDI3 is used through either Avisynth or Vapoursynth in Hybrid, by default Vapoursynth is used.
So you use Vapoursynth.

The image was created using FeMaSR only (no other filters used) which is an ai filter Hybrid does support through it's torch-addon package.
The below was a suggestion on how to deal with the ringing&halos in the image.

Note that most filtering can be pretty destructive, so depending on the selection, settings and content they can easily destroy wanted details.

The code I posted is a combination of YAHR, FineDehalo an GLSL anti-ringing which to me seemed a good choice to remove the halo- and ringing- artifacts while not being too aggressive.

For upscaling there are tons of methods, in Hybrid. NNEDI3 is an upscaler which does not do extensive filtering while upscaling, others (especially ai based stuff) tends to do additional filtering.
When using ai (trained) resizing, I often can help to first see what the resizer will do with the unfiltered content and check what happens if you add filters before the resizer. Additional filtering before the resizing might help, but could also destroy stuff the ai trained model takes as a clue to do stuff. So sometimes feeding a not totally cleaned source can be better than something that is smoothed when zooming into the source.

Cu Selur

Ps.: Since your card should at least be capable of doing some ai based filtering, I'll send you a link to a fresh dev version which links to the ai stuff via pm in a few minutes.
Here are some examples what some of the ai filters do (only using deblockers and denoisers):
SCUNet at default settings:: https://imgsli.com/MTU2MDUx
DPIR (denoise) at default settings: https://imgsli.com/MTU2MDUy
DPIR (deblock) with strength 15: https://imgsli.com/MTU2MDU2
BasicVSR++(interval=1)+YAHR(clip, blur=1, depth=16)+FineDeHalo(): https://imgsli.com/MTU2MDU3
Here also a conventional approach with MCDeGrainSharp+YAHR+FineDehalo: https://imgsli.com/MTU2MDYw

[Al74]

Selur,
I have tried different filters and options and when I get more results I write in detail which method(s) is(are) satisfying for me. Some time ago I used YAHR 1 with Temporal Degrain, but some more tweaks are necessary to improve this path even more. From what I can see now, Hybrid filter chain sequence (YAHR, FineDeHalo, GLSL is fine - I understand you used Filter QUEUE to create this sequence...?)
What is the benefit of YAHR 1 16 bit vs YAHR 1 32 bit)?
Also DPIR Deblock and later on MCDegrain Sharp is also promising, but more MCDegrain Sharp filter settings are necessary to improve footage even more.

[Selur]

What is the benefit of YAHR 1 16 bit vs YAHR 1 32 bit)?

'depth' does not refer to bit depth.
YAHR internally uses aWarphSharp2 and the depth parameter sets depth in aWarphSharp2.
If basically defines how far the filter is active. So a smaller value limits the range around edges that YAHR modifies the output.
So the benefit of using a smaller depth value is that it modifies less of the image. Ideally, you just want to change the halo and not more.

Cu Selur

[Al74]

Selur,
Thank you for the clarification. I see. I shall play with the settings of YAHR filter. All in all now using some different methods the results are promising. DPIR deblock and SCUNet removes halos etc but leave noise BUT later on I apply MCDegrain Sharp.

[Selur]

Be sure to look also at the fine details when using machine learning based tools like DPIR and SCUNet, since those filters sometimes remove quite a bit of tine details alongside with the noise/artifacts.

Cu Selur

[Al74]

Yes, you are right. I just use one or two clips to see the differences. Another thing is to use clips, which, apart from fine details, contain people (faces) and inscriptions, since some filters/AI models just crach them leading to very artificial/unreal results even though other parts of the footage gain a lot. So some compromise is needed but a natural outlook is the most important, I mean true-to-life improved clip.
So that is why I compare YAHR 1 16 bit, FineDeHalo, AntiRing GLSLS and MCDegrain Sharp with different settings vs DPIR/SCUNet + MCDegrain Sharp with different settings. Moreover, only static images from the clips are not enough since there are many problems when one examines the clips in motion.

[Al74]

Be sure to look also at the fine details when using machine learning based tools like DPIR and SCUNet, since those filters sometimes remove quite a bit of tine details alongside with the noise/artifacts.

Cu Selur

Hi Selur and all members,
Since I have not dealt with these AVI clips for looong time, I shall come back soon to check different settings. I remember that FeMaSR introduced very strange artefacts in motion. You showed just an image comparison, but in motion, it was not good. I shall check once more.
Scunet is quite good BUT it removes fine details and makes faces plastiky. Maybe Yahr, Fine Dehalo and GLSL will give a more natural look.
Have you used the default settings of Fine DeHalo?
I shall prepare a more comprehensive comparison and share the results.
Greetings,