Deoldify Vapoursynth filter

[Selur]

I'll look at the changes.
You might want to think about some other way of adding parameters.
ddeoldify is stating to have too many parameters to be usable without a gui.
(maybe group them with namedtuples)

Cu Selur

[Selur]

Updated the download for the dev version (same link).
with default values: https://imgsli.com/MjQ3MzI0

Cu Selur

[Dan64]

Nice test!

I noted that sometime on dark scenes even deoldify is not stable in this case there is not a big advantage to use it to stabilize ddcolor.
So I added another parameter "luma_mask_sat" if this parameter is below 1. The dark pixels in ddcolor will be replaced by the desaturared pixel of deoldify, but the final merge will be performed on a not desaturated version of deoldify.
This should fix some of the problems with LumaMaskMethod.
I attached the updated version.

Dan

[Dan64]

I'm struggling in getting the proper dark color in colored images. 

In images where there is darkness, deoldify/ddcolor not being able to understand what it is the dark area and what color to give it, they often decide to color these dark areas with blue which then becomes red and then returns to blue introducing a psychedelic effect. 

I end-up taking a draconian decision to add a post-process filter called "dark_darkness", that makes dark images really dark (and not flashing blue/red). 
This effect is achieved by de-saturating (via parameter "dark_sat") and reducing the brightness (via parameter "dark_bright")  that parts of the image with a low luma value (below the "dark_threshold").

I attached a new version of the filter.

Thanks,
Dan

[Selur]

Will look at the changes later.
Have you experimented with using something like GrayWorld before applying the colorization?
Tweak(sat=0) + GrayWorld + dd: https://imgsli.com/MjQ3Mzkx
Tweak(sat=0) + Retinext: https://imgsli.com/MjQ3Mzky
(Adjusting the video, maybe just for one of the filters, before using them. Maybe changing to with spefific color temperature after the filtering and before the merging?)

Cu Selur

[Dan64]

I will release a new updated version, where both the LumaMaskedMerge and the DarkDarkness filter will use a gradient merge instead of a simple b&w mask. 
This should solve almost all the problems.

[Selur]

Nice, I'll wait for that before I adjust Hybrid then.
Also if you found a merge option that works for everything, maybe dropping some of the not useful merge options?

Cu Selur

Ps.: for amusement https://forum.videohelp.com/threads/3771...ost2728069

[Dan64]

The sample selected is in a bright situation, the worst happen on dark scenes.

I updated the filter introducing a "gradient mask" merge.

This merge is controlled by 2 parameters: dark_luma, white_luma
Merging img1 and img2 with this method: till the luma of img2 < dark_luma, will be copied the prixels from img1, for luma > white_luma will be copied the pixels from img2.
In between the pixels of img1 and img2 are weighted summed with the weight that will move from 1 (only pixels of img1) to 0 (only pixels of img2).

The ddeoldify function is

def ddeoldify(
    clip: vs.VideoNode, model: int = 0, render_factor: int = 24, sat: list = [1.0,1.0], hue: list = [0.0,0.0], dd_model: int = 1, dd_render_factor: int = 24,
    dd_tweak_luma_bind: list = [False, 0.4, 0.4], dd_bright: float = 0, dd_cont: float = 1, dd_gamma: float = 1, dd_method: int = 2, ddcolor_weight: float = 0.5,
    dd_method_params: list = [0.6, 2.0, 0.15, 0.2, False], chroma_resize: bool = True, luma_mask: list = [0.3, 0.6, 1.0], dark_darkness: list = [False, 0.3, 0.3, 0.2, -0.05], device_index: int = 0, n_threads: int = 8, dd_num_streams: int = 1, torch_hub_dir: str = model_dir

respect to the version 2024-03-14, I applied the following changes:

1) the parameters luma_mask_limit, luma_mask_sat has been include in the list luma_mask[] with the added parameter luma_white_limit (equivalent of white_luma)
2) the list dd_method_params[] has been extended with the boolean parameter invert_clips (if true invert the clip order in the Merge methods)
3) the list dark_darkness has been extended with the parameter white_threshold

more info are in __init__.py

I attached the source code.

Dan

[Selur]

download contains vs-deoldify-1.1.15

__version__ = "1.1.5"


package_dir = os.path.dirname(os.path.realpath(__file__))
model_dir = os.path.join(package_dir, "models")

#configuring torch
torch.backends.cudnn.benchmark=True


import vapoursynth as vs

def ddeoldify(
    clip: vs.VideoNode, model: int = 0, render_factor: int = 24, sat: list = [1,1], hue: list = [0,0], dd_method: int = 0, dd_weight: float = 0.5,
    dd_strength: int = 24, dd_model: int = 0, device_index: int = 0, n_threads: int = 8, dd_num_streams: int = 1, torch_hub_dir: str = model_dir
) -> vs.VideoNode:

[Dan64]

Sorry, I zipped the wrong folder.
I attached the correct version.

Dan