Hello Selur,
I completed the changes for the new version 3.0.0.
On filtering side I improved the "chroma stabilizer" to solve the problem related to gray areas in the past/future frames.
I also added the requested changes on ddcolor regrading the optional fp16 and delayed loading of the filter.
But the big changes are on the interface of the filter that has changed significantly.
As agreed I removed the post-process stabilization filters in another filter called:
ddeoldify_stabilizer
The new header for
ddeoldify is
Code:
def ddeoldify(
clip: vs.VideoNode, color_method: int = 2, merge_weight: float = 0.4, deoldify_params: list = [0, 24, 1.0, 0.0], ddcolor_params: list = [1, 24, 1.0, 0.0], ddcolor_tweaks: list = [False, 0.4, 0.4, 0.0, 1.0, 1.0, True, 1], combine_params: list = [False, 0.6, 2.0, 0.15, 0.2, 0.3, 0.6, 1.0], darkness: list = [False, 0.2, 0.8], chroma_resize: bool = True, device_index: int = 0, np_threads: int = 8, torch_hub_dir: str = model_dir) -> vs.VideoNode:
"""A Deep Learning based project for colorizing and restoring old images and video using Deoldify and DDColor
:param clip: clip to process, only RGB24 format is supported.
:param color_method: method used to combine deoldify() with ddcolor() (default = 2):
0 : deoldify only (no merge)
1 : ddcolor only (no merge)
2 : Simple Merge:
the images are combined using a weighted merge, where the parameter merge_weight
represent the weight assigned to the colors provided by ddcolor()
3 : Adaptive Luma Merge:
given that the ddcolor() perfomance is quite bad on dark scenes, the images are
combined by decreasing the weight assigned to ddcolor() when the luma is
below a given threshold given by: luma_threshold.
For example with: luma_threshold = 0.6 and alpha = 1, the weight assigned to
ddcolor() will start to decrease linearly when the luma < 60% till "min_weight".
For alpha=2, begins to decrease quadratically.
4 : Constrained Chroma Merge:
given that the colors provided by deoldify() are more conservative and stable
than the colors obtained with ddcolor() images are combined by assigning
a limit to the amount of difference in chroma values between deoldify() and
ddcolor() this limit is defined by the parameter threshold. The limit is applied
to the image converted to "YUV". For example when threshold=0.1, the chroma
values "U","V" of ddcolor() image will be constrained to have an absolute
percentage difference respect to "U","V" provided by deoldify() not higher than 10%
5 : Luma Masked Merge:
the clips are combined using a mask merge, the pixels of ddcolor() with luma < luma_mask_limit
will be filled with the pixels of deoldify(), if the parameter merge_weight > 0
the final masked image will be merged again with deoldify()
:param merge_weight: weight given to ddcolor's clip in all merge methods, range [0-1] (0.01=1%)
:param deoldify_params: parameters for deoldify():
[0] deoldify model to use (default = 0):
0 = ColorizeVideo_gen
1 = ColorizeStable_gen
2 = ColorizeArtistic_gen
[1] render factor for the model, range: 10-44 (default = 24).
[2] saturation parameter to apply to deoldify color model (default = 1)
[3] hue parameter to apply to deoldify color model (default = 1)
:param ddcolor_params: parameters for ddcolor():
[0] ddcolor model (default = 1):
0 = ddcolor_modelscope,
1 = ddcolor_artistic
[1] render factor for the model, if = 0 will be auto selected
(default = 24) [range: 0, 10-64]
[2] saturation parameter to apply to deoldify color model (default = 1)
[3] hue parameter to apply to deoldify color model (default = 1)
[4] enable FP16 inference in DDColor
:param ddcolor_tweaks: tweak parameters for ddcolor():
[0] : luma_constrained_tweak -> luma constrained ddcolor preprocess enabled (default = False), range: [True, False]
when enaabled the average luma of a video clip will be forced to don't be below the value
defined by the parameter "luma_min". The function allow to modify the gamma
of the clip if the average luma is below the parameter "gamma_luma_min"
[1] : luma_min -> luma (%) min value for tweak activation (default = 0, non activation), range [0-1]
[2] : gamma_luma_min -> luma (%) min value for gamma tweak activation (default = 0, non activation), range [0-1]
[3] : ddcolor tweak's bright (default = 0)
[4] : ddcolor tweak's constrast (default = 1)
[5] : ddcolor tweak's gamma (default = 1)
[6] : enable/disable FP16 in ddcolor inference
[7] : ddcolor's number of CUDA streams to enqueue the kernels (default = 1) (default = 1)
:param combine_params: list with the parameters used by combining methods:
[0] : if true invert the clip order in all the combining methods
-parameters for method: "Adaptive Luma Merge" (method=3)
[1] : luma_threshold, used by: AdaptiveLumaMerge, range [0-1] (0.01=1%)
[2] : alpha (float), used by: AdaptiveLumaMerge, range [>0]
[3] : min_weight, used by: AdaptiveLumaMerge, range [0-1] (0.01=1%)
-parameters for method: "Constrained Chroma Merge" (method=4)
[4] : chroma_threshold (%), used by: ConstrainedChromaMerge range [0-1] (0.01=1%)
-parameters for method: "Luma Masked Merge" (method=5)
[5] : luma_mask_limit, luma limit for build the mask used in Luma Masked Merge, range [0-1] (0.01=1%)
[6] : luma_white_limit, if > luma_mask_limit will be applied a gradient till luma_white_limit, range [0-1] (0.01=1%)
[7] : luma_mask_sat: if < 1 the ddcolor dark pixels will substitute with the desaturated deoldify pixels, range [0-1] (0.01=1%)
:param darkness: parameters for darken the clip's dark portions, which sometimes are wrongly colored by the color models\
[0] : darkness_enabled (bool), if true the filter is enabled
[1] : dark_threshold, luma threshold to select the dark area, range [0.1-0.35] (0.01=1%)
[3] : dark_amount: amount of desaturation to apply to the dark area, range [0-1]
:param chroma_resize: if True will be enabled the chroma_resize: the cololorization will be applied to a clip with the same
size used for the models inference(), but the final resolution will be the one of the original clip.
:param device_index: device ordinal of the GPU, choices: GPU0...GPU7, CPU=99 (default = 0)
:param np_threads: number of threads used by numpy, range: 1-32 (default = 8)
:param torch_hub_dir: torch hub dir location, default is model directory,
if set to None will switch to torch cache dir.
"""
As you can see the main changes applied are:
1) The main parameters of deoldify/ddcolor are now grouped in separated lists.
2) All the tweaks parameters of ddcolor (including the fp16 flag) has been included in a dedicated list.
3) All the parameters used by the combination methods are also grouped in a dedicated list
4) "dd_mthod" has been renamed in "color_method"
5) the filter "darkness" now has really the meaning to dark more the dark areas, while the "chroma smoothing" version has been moved in ddeoldify_stabilizer
The header for
ddeoldify_stabilizer is
Code:
def ddeoldify_stabilizer(clip: vs.VideoNode, chroma_smoothing: list = [False, 0.1, 0.2, 0.5, 0.8], chroma_stabilizer: list = [True, 5, 'arithmetic', 1, 60, 0.5, 0.8, 0], chroma_limiter: list = [False, 0.02], chroma_resize: list = [True, 24]) -> vs.VideoNode:
"""Video color stabilization filter, which can be applied to stabilize the chroma components in ddeoldify colored clips.
:param clip: clip to process, only RGB24 format is supported.
:param chroma_smoothing: parameters to adjust the saturation and "vibrancy" of the clip.
[0] : chroma_smoothing_enabled (bool), if true the filter is enabled
[1] : dark_threshold, luma threshold to select the dark area, range [0-1] (0.01=1%)
[2] : white_threshold, if > dark_threshold will be applied a gradient till white_threshold, range [0-1] (0.01=1%)
[3] : dark_sat: amount of desaturation to apply to the dark area, range [0-1]
[4] : dark_bright (float): darkness parameter it used to reduce the "V" component in "HSV" colorspace, range [0, 1]
:param chroma_stabilizer: parameters for the temporal color stabilizer
[0] : colstab_enabled (bool), if true the filter will be enabled
[1] : colstab_nframes, number of frames to be used in the stabilizer, range[3-15]
[2] : colstab_mode, type of average used by the stabilizer: range['arithmetic', 'weighted']
[3] : colstab_sat: saturation applied to the restored gray prixels [0,1]
[4] : colstab_tht, threshold to detect gray pixels, range [0,235], if=0 is not applied the restore
[5] : colstab_weight, weight to blend the restored imaage, range [0,1], if=0 is not applied the blending
[6] : colstab_tht_scen, threshold for scene change detection (default = 0.8), range [0,1]
[7] : colstab_algo, algo used for stabilization: 0=clips (default), 1=frames. It is an
internal parameter added for internal testing, algo=0 is 40%-60% faster and should be always used.
:param chroma_limiter: parameters for the temporal color limiter
[0] : colimit_enabled (bool), if true the filter will be applied after the merge of Deoldify and DDColor
[1] : colimit_deviation, chroma of current frame will be forced to be inside the range defined by "deviation", range[0.01-0.5]
:param chroma_resize: parameters for the chroma resizer pre-filter
[0] : if True the chroma resizer is enabled (can improve the filters speed significantly)
[1] : render_factor to apply to the filters, the frame size will be the same used for the models inference(),
but the final resolution will be the one of the original clip.
In the filter
chroma_stabilizer the last parameter
algo should not be included in the GUI since has been added for internal testing.
This filter is the only enabled by default, because is the more effective in stabilizing the colors
I hope that you will be able to apply the changes in Hybrid.
Thanks and Happy Easter,
Dan
I small update (no changes on filters headers)
Dan
