Deoldify Vapoursynth filter

[Selur]

Got another question: If mask is used, does sc_threshold still look at the whole image or just the mask ? ( I think the later might be the right choice, but I'm unsure whether 0.1 isn't a too low then)

[Dan64]

Actually I developed a version of ProPainter with the parameter "sc_on_region", in the case this parameter is True, the SCDetect will be applied only on the mask_region.
In this case the sc_threshold need to be set to a lower value, because usually the logo will cover most of mask region and the change in the mask_region are less than on full image.

I decided to don't deliver it becasuse on my tests this version was slower.

If you are interested in testing it I can provide you this version.

Dan

---

HAVC_deepex:

:param sc_framedir:         if set, define the directory where are stored the reference frames. If only_ref_frames=True,
                                and method=0 this directory will be written with the reference frames used by the filter.
                                if method!=0 the directory will be read to create the reference frames that will be used
                                by "Deep-Exemplar-based Video Colorization".

:param sc_framedir:         If set, define the directory where the reference frames are stored.
                                The reference frame name must be in the format: ref_nnnnnn.[jpg|png|jpeg], for example
                                the reference frame 897 must be named: ref_000897.[jpg|png|jpeg]

=> should this be one text?

I will release a new RC version with this comment fixed and a change in a parameter name, which will allow to control another interesting feature, that I'm still testing.
Dan

[Selur]

I decided to don't deliver it becasuse on my tests this version was slower.

I suspect it will be slower if more scene changes are detected. + I think this option should be added.

In this case the sc_threshold need to be set to a lower value, because usually the logo will cover most of mask region and the change in the mask_region are less than on full

from the argumentation, I would have gone for higher values for the threshold, not lower ones

I will release a new RC version with this comment fixed and a change in a parameter name, which will allow to control another interesting feature, that I'm still testing.

Looking forward to it, probably won't get around to test it until tomorrow evening. (visiting parents tomorrow after work)

Cu Selur

[Dan64]

Hello Selur,

  I released the new RC3 (attached) and I hope that it will be the final one.

 The main changes are:

1) in HAVC_deepex() the parameter "enable_resize" has been replaced by the parameter "ref_merge"
    With the following description:
    

Method used by DeepEx to merge the reference frames with the frames propagated by DeepEx.
 It is applicable only with DeepEx method: 0, 1, 2.
 The HAVC reference frames must be produced with frequency = 1.
 Allowed values are:
        0 = No RF merge (reference frames can be produced with any frequency)
        1 = RF-Merge Low (reference frames are merged with low weight)
        2 = RF-Merge Med. (reference frames are merged with medium weight)
        3 = RF-Merge High (reference frames are merged with high weight)

  
  the tricky part is that this method (when > 0) requires that in the function HAVC_ddeoldify the parameter "sc_min_freq" is set = 1.
  Since this parameter is included in a separate function I cannot enforce it, in this case should be Hybrid to set sc_min_freq=1 when  ref_merge  > 0.

2) in HAVC_main I added the parameter DeepExRefMerge to manage the value of parameter "ref_merge" in HAVC_deepex() (it is just a propagation)

    This method has been added to fix the problem reported in post: #500

    The results are quite satisfatory, here the frame 20 colored using DeepEx



   the image was obtained using the following code:

clip = HAVC_main(clip, Preset='Fast', ColorFix='Violet/Red', ColorTune='Light', EnableDeepEx=True, DeepExMethod=1, DeepExPreset='Medium', DeepExRefMerge=2, ScFrameDir="D:/Tests/Green/ref_color")

   
 and the following reference image



 I hope you have time to adapt the GUI.

Thanks,
Dan

[Selur]

today updated latest ver. not working.

Are you referring to vsdeoldify-4.0.0_RC3 or Hybrid 2024.06.09.1 ?

[Selur]

No proper details. => don't care + wrong thread to post about problems with released versions.

[Selur]

@Dan64: uploaded a new deoldify test version.
https://imgsli.com/MjcwOTc4


Cu Selur

[Dan64]

Your reference image was too dark.

I increased the brightness of reference image and obtained the following results:  https://imgsli.com/MjcwOTk0

in my example the color of the raincoats is very similar to that of the reference image.

in this case the reference image must be named ref_000000.png otherwise is not recognized.

Dan

[Selur]

Nice.

[Selur]

I noticed a bug in Hybrid itself.
When the current color sampling was RGB (created from YUV with a color matrix that was not the required by the filter), Hybrid crashed.
ColorMatrix changes can only be applied in YUV color space, thus changing the color matrix on content that is already RGB does not really make sense. (Hybrid only needs to change which colorMatrix to use on the next YUV<>RGB conversion)
=> updated test download

Cu Selur