Deoldify Vapoursynth filter

[Dan64]

In order to fix the problems related to AverageFrame.

I'm planning to:

1) limit to 11 frames the max number of frames to be used in the average
2) enable only "central" average because left/right are complementary
3) introduce a weighted average where the near frames have an higher weight.

Here an example with 11 frames

https://imgsli.com/MjQ5Mzc3

This should almost fix the observed problems.

Dan

[Selur]

In your comparisions it would help, if you also added an image with the none weighted result, so see whether it even is a good idea to weight the results.

enable only "central" average because left/right are complementary

Okay, so 'colstab_mode' will be removed? (I'll at least remove it from the gui, since there's nothing to choose. )


Cu Selur
Ps.: Will adjust Hybrid when a new vs-deoldify version is available.

[Dan64]

No mode will be still available, with range['arithmetic', 'weighted'] (default = ' weighted ')

Dan

[Selur]

Ah, okay.
Will adjust Hybrid when the time comes.

Cu Selur

[Dan64]

I'm going to release the attached version.

Respect to the previous version I applied the following changes:

1) "Mode" in "Chroma Stabilization" will accept only values:"arithmetic" (default), "weighted"
2) the max frames allowed in  "Chroma Stabilization" now is 15
3) is not more necessary the plugin: "MiscFilters.dll"  (will be used the standard Vapoursynth plugin)
4) changed some default value (see the ddeoldify() function header)
5) added new post-process filter "Chroma Limiter"

The new filter "Chroma Limiter"  will force the chroma chenges in a frame to be below the chroma deviation limit (default 2%) of previous frame.

The behavior of this new filter is quite satisfactory, see the comparison: https://imgsli.com/MjQ5NTIw  

Moreover in the vsdeoldify package I exported 2 new filters extracted from ddeoldify():

#1 Chroma resize filter, derived from ddeoldify
dd_chroma_resize(clip_hires: vs.VideoNode, clip_lowres: vs.VideoNode) -> vs.VideoNode:

and

#2 Video color stabilization filter, derived from ddeoldify
dd_video_stabilizer(clip: vs.VideoNode, chroma_resize: list = [False, 32], color_smoothing: list = [False, 0.1, 0.2, 0.5, 0.8], color_stabilizer: list = [False, 5, 'arithmetic', 1, True], color_limiter: list = [False, 0.02]) -> vs.VideoNode: 

The first filter allows to transfer the chroma components from a low resolution movie to the same high resolution movie (useful to speed up filters working on the chroma components).
The second filter is including all the post-process filters developed for ddeoldify() and could be useful to restore movies colored with others applications.

It's up to you to decide if it's worth including them in Hybrid .

Dan

[Dan64]

I released the new version: https://github.com/dan64/vs-deoldify/rel...tag/v2.0.1

For for compatibility with the previous version, the mode "center" is mapped to "arithmetic" and "left"/"right" to "weighted".
So that this update can also work with the current Hybrid version.

Dan

[Selur]

I'll release a new Hybrid version tomorrow.
(to drunk today - 'Steak Day' with a friend today)

Cu Selur

[Dan64]

Hello Selur,

  sorry to bother you but I have a problem with Vapoursynth that I can't understand.

  In order to improve the "chroma limiter", I was thinking to add the possibility to repeat the filter more times (max 5).
  To do that I wrote this piece of code

def _clip_chroma_limiter_ex(clip: vs.VideoNode = None, deviation: float = 0.05, steps: int = 1, tht: int = 10) -> vs.VideoNode:
   
    max_steps = max(min(steps, 5), 1)
       
    clip_limited = vs_clip_chroma_stabilizer(clip, deviation=deviation)
    for i in range(1, max_steps):
        clip_limited=vs_clip_chroma_stabilizer(clip_limited, deviation=deviation)
   
    if tht < 1 or tht > 255:
        return clip_limited

    # calculate motion mask
    clipMask = clip
    clipMask = clipMask.resize.Bicubic(format=vs.GRAY8, matrix_s="470bg", range_s="limited")
    clipMask = vs.core.motionmask.MotionMask(clip=clipMask, th1=tht, th2=tht, tht=tht) # pixels with abs(diff) < tht will be black (static parts)
    clipMask = vs.core.std.InvertMask(clip=clipMask) # invert so that static parts are white (weight=1)
    # merge in YUV color space       
    clipMask = clipMask.resize.Bicubic(format=vs.YUV444PS, matrix_s="470bg", range_s="limited")
    clip_limited = clip_limited.resize.Bicubic(format=vs.YUV444PS, matrix_s="470bg", range_s="limited")
    clip = clip.resize.Bicubic(format=vs.YUV444PS, matrix_s="470bg", range_s="limited")
    clip = vs.core.std.MaskedMerge(clipa=clip, clipb=clip_limited, mask=clipMask) # MotionMask
    # restore RBG24 color space
    clip = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="limited", dither_type="error_diffusion")
   
    return clip

Now the problem:

If I set "tht=0" the motion mask is not applied and I can see the filter working:

Here with steps=1



Here with steps=5



But if I set "tht" with a value > 1 and < 255, independently from the value I get always the same image:



like the Mask was always filled with weights=0

Do you have any idea on the reason ?

Dan

[Selur]

Side note, looking at:

clip_limited = vs_clip_chroma_stabilizer(clip, deviation=deviation)
    for i in range(1, max_steps):
        clip_limited=vs_clip_chroma_stabilizer(clip_limited, deviation=deviation)

shouldn't you use 'range(1, max_steps-1)', so that max_steps sets how often the filter is applied ?
Otherwise for max_steps the filter would be applied 'max_steps +1 '-times.

Now the problem:

If I set "tht=0" the motion mask is not applied and I can see the filter working:

Here with steps=1

Okay, what is the problem, atm.

if tht < 1 or tht > 255:
        return clip_limited

would return the 'max_steps +1 ' vs_clip_chroma_stabilizer without the motion masking.
Shouldn't "if tht < 1 or tht > 255" be "if tht < 9 or tht > 255" ?


Cu Selur

[Selur]

Ah, post wasn't finished. (Images were just shown without explaination as attachment)