Selur's Little Message Board
[HELP] Anti Aliasing filters & Resize!! - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Hybrid - Support (https://forum.selur.net/forum-1.html)
+--- Forum: Problems & Questions (https://forum.selur.net/forum-3.html)
+--- Thread: [HELP] Anti Aliasing filters & Resize!! (/thread-1641.html)

Anti Aliasing filters & Resize!! - Cauptain - 17.12.2020

Hello Selur, how are you?

I hope you are well in this epidemy.

My friend, help me. I have been using Hibryd for a long time and I never had any difficulty with the filters used.

However I decided to use the anti Aliasing filters and now I need your help.

I have a 3D video (video game) that by default is 900x558 and I would like to use the best AA filter and the best Resize algorithm for 1080p (No AR)

My question: What is the best AA filter and the best RESIZE and how to set up to get the best quality from both?


Video used: https://drive.google.com/file/d/12wpvRHIPE962o8cB_CRNzmBlodTG7lf6/view?usp=sharing


Claudio

RE: Anti Aliasing filters & Resize!! - Selur - 17.12.2020

"by default is 900x558 "
I doubt that this is the native resolution of the game, probably more like 320x240 or 400x240 or something like that, also there are tons of compression artifacts.
Problem is anything that does get rid of the aliasing would have to smooth a lot and kill tons of details.
-> not sure this is doable in a meaningful way (anti aliasing should be done in the emulator or the game,..)

Instead of trying AA filters better try aWarpSharp2.

Cu Selur

RE: Anti Aliasing filters & Resize!! - Selur - 17.12.2020

If this is really the best source you have, try:
  1. for AA:
    • enable 'Filtering->Vapoursynth->Line->AntiAliasing->AAF'
    • enable 'Filtering->Vapoursynth->Custom->Insert Before->AAF' and insert:
      clip = core.std.Transpose(clip=clip)
      clip = havsfunc.aaf(inputClip=clip)
      clip = core.std.Transpose(clip=clip)
      clip = havsfunc.aaf(inputClip=clip)
      clip = core.std.Transpose(clip=clip)
      clip = havsfunc.aaf(inputClip=clip)
      clip = core.std.Transpose(clip=clip)
      this might be a bit overkill, but the general idea is to apply the AntiAliasing on the normal video and the transposed video.
  2. to get some contrast back:
    1. enable 'Filtering->Vapoursynth->Sharpen->CAS'
    2. set 'Filtering->Vapoursynth->Sharpen->CAS->Sharpness' to '1'
  3. for the resizing to 1080p:
    1. Change 'Filtering->Vapoursynth->Misc->Filter Order' by moving 'Resize' below 'AAF'
    2. enable 'Filtering->Crop/Resize->Base->Picture Crop' and set 'bottom' to: 24 and 'right' to:4
    3. enable 'Filtering->Crop/Resize->Base->Resize'
    4. set 'Filtering->Crop/Resize->Base->Resize->Auto adjust' to 'width'
    5. set 'Filtering->Crop/Resize->Base->Resize->Target resolution' to 1080
    6. enable 'Filtering->Crop/Resize->Base->Letterbox'
    7. set 'Filtering->Crop/Resize->Base->Letterbox->Width' to 1920
    8. personally I would not use any sepcial resizer, but if you think it makes a noticeable difference use 'Filtering->Vapoursynth->Resize->NNEDI3' and enable 'GPU' if your hardware supports it.
otherwise if possible let the emulator or the game to the AA if possible, since the above does look 'okay' on still image, but I'm not sure how well it looks during normal playback.

Cu Selur

RE: Anti Aliasing filters & Resize!! - Selur - 17.12.2020

Quote:(No AR)
Augmented Reality?

Cu Selur

RE: Anti Aliasing filters & Resize!! - Cauptain - 17.12.2020

(17.12.2020, 18:02)Selur Wrote: "by default is 900x558 "
I doubt that this is the native resolution of the game, probably more like 320x240 or 400x240 or something like that, also there are tons of compression artifacts.
Problem is anything that does get rid of the aliasing would have to smooth a lot and kill tons of details.
-> not sure this is doable in a meaningful way (anti aliasing should be done in the emulator or the game,..)

Instead of trying AA filters better try aWarpSharp2.

Cu Selur

Its a Youtube video. The guy record using a png template.

Will try now your settings.

Many thanks.


Claudio
(17.12.2020, 19:20)Selur Wrote:
Quote:(No AR)
Augmented Reality?

Cu Selur

Resize with No Aspect Ratio (Full screen)  because strange resolution.  Big Grin


Claudio

RE: Anti Aliasing filters & Resize!! - Selur - 18.12.2020

Quote:Will try now your settings.
Did you try it? How does it look in motion? (I only looked it frame by frame and I fear that the AA will cause flickering in motion.)
(didn't have time to actually encode the script to something and watch it in motion)

Also as further explanation, the
Quote:'Filtering->Vapoursynth->Custom->Insert Before->AAF' and insert:
clip = core.std.Transpose(clip=clip)
clip = havsfunc.aaf(inputClip=clip)
clip = core.std.Transpose(clip=clip)
clip = havsfunc.aaf(inputClip=clip)
clip = core.std.Transpose(clip=clip)
clip = havsfunc.aaf(inputClip=clip)
clip = core.std.Transpose(clip=clip)
part of the script followed be the normal "clip = havsfunc.aaf(inputClip=clip)" does simply apply:
1. aaf on the transposed video
2. aaf on the normal video
3. aaf on the transposed video
4. aaf on the normal video
you could do similar stuff with other AA filters or use aaf less frequent if the AA effect is too strong.

Cu Selur

RE: Anti Aliasing filters & Resize!! - Cauptain - 01.09.2021

(18.12.2020, 22:29)Selur Wrote:
Quote:Will try now your settings.
Did you try it? How does it look in motion? (I only looked it frame by frame and I fear that the AA will cause flickering in motion.)
(didn't have time to actually encode the script to something and watch it in motion)

Also as further explanation, the
Quote:'Filtering->Vapoursynth->Custom->Insert Before->AAF' and insert:
clip = core.std.Transpose(clip=clip)
clip = havsfunc.aaf(inputClip=clip)
clip = core.std.Transpose(clip=clip)
clip = havsfunc.aaf(inputClip=clip)
clip = core.std.Transpose(clip=clip)
clip = havsfunc.aaf(inputClip=clip)
clip = core.std.Transpose(clip=clip)
part of the script followed be the normal "clip = havsfunc.aaf(inputClip=clip)" does simply apply:
1. aaf on the transposed video
2. aaf on the normal video
3. aaf on the transposed video
4. aaf on the normal video
you could do similar stuff with other AA filters or use aaf less frequent if the AA effect is too strong.

Cu Selur

Hi Selur,

Tested it previously and it worked very well.

Saturday I will be releasing a new pack of games and used your method again on videos and it served me very, very well indeed.

I did what you said about testing with fewer "lines parameters" and preferred the "full method".

He does a lot better than I expected.

Thank you very much for your attention to the answers.


Claudio