+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: Deoldify Vapoursynth filter (/thread-3595.html)
RE: Deoldify Vapoursynth filter - Dan64 - 25.05.2024
You are right. In effect I installed the packages in Hybrid with non version number and let pip to select the most appropriate version.
The version numbers that I included were the versions installed by pip.
opencv-contrib-python==4.9.0.80As you know these AI programs require a lot of packages, I had to figure out what were the missing packages in Hybrid.
Fortunately were only 3, with opencv-contrib-python being the most important.
Dan
RE: Deoldify Vapoursynth filter - Selur - 26.05.2024
old:
def ddeoldify_main(clip: vs.VideoNode, Preset: str = 'Fast', VideoTune: str = 'Stable', ColorFix: str = 'Violet/Red',
ColorTune: str = 'Light', ColorMap: str = 'None', degrain_strength: int = 0,
enable_fp16: bool = True) -> vs.VideoNode:def ddeoldify_stabilizer(clip: vs.VideoNode, dark: bool = False, dark_p: list = [0.2, 0.8], smooth: bool = False, smooth_p: list = [0.3, 0.7, 0.9, 0.0, "none"],
stab: bool = False, stab_p: list = [5, 'A', 1, 15, 0.2, 0.15], colormap: str = "none", render_factor: int = 24) -> vs.VideoNdef ddeoldify(clip: vs.VideoNode, method: int = 2, mweight: float = 0.4, deoldify_p: list = [0, 24, 1.0, 0.0],
ddcolor_p: list = [1, 24, 1.0, 0.0, True],
dotweak: bool = False, dotweak_p: list = [0.0, 1.0, 1.0, False, 0.2, 0.5, 1.5, 0.5],
ddtweak: bool = False, ddtweak_p: list = [0.0, 1.0, 2.5, True, 0.3, 0.6, 1.5, 0.5, "300:360|0.8,0.1"],
degrain_strength: int = 0, cmc_tresh: float = 0.2, lmm_p: list = [0.2, 0.8, 1.0],
alm_p: list = [0.8, 1.0, 0.15], cmb_sw: bool = False, device_index: int = 0,
torch_dir: str = model_dir) -> vs.VideoNode:def HAVC_main(clip: vs.VideoNode, Preset: str = 'Fast', VideoTune: str = 'Stable', ColorFix: str = 'Violet/Red',
ColorTune: str = 'Light', ColorMap: str = 'None', EnableDeepEx: bool = False, DeepExMethod: int = 0,
DeepExPreset: str = 'Fast', ScFrameDir: str = None, ScThreshold: float = 0.1, ScMinFreq: int = 0,
enable_fp16: bool = True) -> vs.VideoNode:def HAVC_deepex(clip: vs.VideoNode = None, clip_ref: vs.VideoNode = None, method: int = 0, render_speed: char = 'medium', render_vivid: bool = True,
enable_resize : bool = False, sc_framedir: str = None, only_ref_frames : bool = False, dark: bool = False, dark_p: list = [0.2, 0.8], smooth: bool = False, smooth_p: list = [0.3, 0.7, 0.9, 0.0, "none"],
colormap: str = "none") -> vs.VideoNode:def HAVC_ddeoldify(
clip: vs.VideoNode, method: int = 2, mweight: float = 0.4, deoldify_p: list = [0, 24, 1.0, 0.0],
ddcolor_p: list = [1, 24, 1.0, 0.0, True], ddtweak: bool = False,
ddtweak_p: list = [0.0, 1.0, 2.5, True, 0.3, 0.6, 1.5, 0.5, "300:360|0.8,0.1"],
cmc_tresh: float = 0.2, lmm_p: list = [0.2, 0.8, 1.0], alm_p: list = [0.8, 1.0, 0.15], cmb_sw: bool = False,
sc_threshold: float = 0.0, sc_min_freq: int =0, device_index: int = 0, torch_dir: str = model_dir) -> vs.VideoNode:def HAVC_stabilizer(clip: vs.VideoNode, dark: bool = False, dark_p: list = [0.2, 0.8], smooth: bool = False,
smooth_p: list = [0.3, 0.7, 0.9, 0.0, "none"], stab: bool = False,
stab_p: list = [5, 'A', 1, 15, 0.2, 0.15], colormap: str = "none",
render_factor: int = 24) -> vs.VideoNode:What happened to 'degrain_strength' ?
Why does HAVC_ddeoldify have sc_threshold and sc_min_freq but not EnableDeepEx and DeepExMethod?
So DeepEx can only be used when using Presets ? (no custom ? Should Hybrid drop 'custom' ?)
Also: I don't see how to properly support DeepExMethod != 0 in Hybrid.
RE: Deoldify Vapoursynth filter - Dan64 - 26.05.2024
I released a new version RC2 with a small update (see attachment).
I tried to be compatible as much as possible with the previous version 3.5.x
The main idea is to use HAVC_deolddify as a source of "exemplar" frames for "Deep-Exemplar-based Video Colorization" but at the same time to keep the use of HAVC_deolddify in "standalone" mode as in the version 3.5.x. Also HAVC_deepex can using HAVC_deolddify as source or can work in "standalone" using only external "exemplar" frames. It is implemented (in RC2) also a mixed mode, where HAVC_deepex can use as sources both HAVC_deolddify and external "exemplar" frames.
I not only renamed the old filter functions (even if the old one are still available but are "deprecated") but for some of them I added/removed some parameters.
With the only exception of HAVC_stabilizer, which is the same as in version 3.5.x and need to be the last function in the rendering pipeline, I applied the following changes:
HAVC_ddeoldify
removed: dotweak, dotweak_p, degrain_strength (KNLMeansCL.dll not more necessary to be loaded)
added: sc_threshold, sc_min_freq (is necessary to load MiscFilters.dll)
if sc_threshold==0 no reference frames are generated and HAVC_ddeoldify will work in "standalone" mode as in version 3.5.x, so is not necessary to add a flag EnableDeepEx
HAVC_main
removed: degrain_strength
added: EnableDeepEx, DeepExMethod, DeepExPreset, ScFrameDir, ScThreshold, ScMinFreq
The "custom" Preset need to be still supported, with the following logic:
If EnableDeepEx == False
The behavior is similar to the one already implemented in 3.5.x
If EnableDeepEx == True
There are 2 cases (introduced in RC2)
If DeepExMethod in (0,1,2)
in the "custom" section need to be enabled the parameters of HAVC_ddeoldify and HAVC_deepex
in this case the rendering pipeline will be:
clip_ref = HAVC_ddeoldify(clip, method=2, mweight=ddcolor_weight[w_id],
deoldify_p=[0, deoldify_rf, 1.0, 0.0], ddcolor_p=[1, ddcolor_rf, 1.0, 0.0, enable_fp16],
ddtweak=True, ddtweak_p=[0.0, 1.0, 2.5, True, 0.3, 0.6, 1.5, 0.5, hue_range],
sc_threshold=ScThreshold, sc_min_freq=ScMinFreq)
clip_colored = HAVC_deepex(clip=clip, clip_ref=clip_ref, method=DeepExMethod, render_speed=DeepExPreset, render_vivid=True,
enable_resize=False, sc_framedir=None, only_ref_frames=False, dark=True, dark_p=[0.2, 0.8],
smooth=True, smooth_p=[0.3, 0.7, 0.9, 0.0, "none"], colormap=chroma_adjust)If DeepExMethod in (3,4)
HAVC_deepex will work in "standalone" using only the external reference frames (clip_ref = None) and in this case in the custom section need to be enabled only the parameters for HAVC_deepex
clip_colored = HAVC_deepex(clip=clip, clip_ref=None, method=DeepExMethod, render_speed=DeepExPreset, render_vivid=True,
enable_resize=False, sc_framedir=ScFrameDir, only_ref_frames=False, dark=True, dark_p=[0.2, 0.8],
smooth=True, smooth_p=[0.3, 0.7, 0.9, 0.0, "none"], colormap=chroma_adjust)see the code of HAVC_main for a full example.
There is an interesting option implemented in HAVC_deepex that is "only_ref_frames" (available only in "custom" mode). If this option is enabled, in the folder specified by the parameter "sc_framedir" will be saved the reference frames used by HAVC_deepex , since this option is added mainly for debugging in this case the clip is not colored by HAVC_deepex and it will be still in black and white and only the reference frames will be colored. This option is useful not only for debug but also to manually adjust the bad colored frames. In this case it is possible to put in another directory only the bad frames with the fixed colors and eventually new reference frames and use the DeepExMethod=1 and set "sc_framedir" with the name of new folder containing the adjusted frames, to regenerate the video containing the "improved" frames. This feature was not available in version 3.5.x and thanks to "Deep-Exemplar-based Video Colorization" it is also possible to perform some kind of semi-automatic colorization.
Dan
RE: Deoldify Vapoursynth filter - Selur - 26.05.2024
Okay,...
I do not seem get the whole 'DeepExMethod' thing.
:param DeepExMethod: Method to use to generate reference frames.
0 = HAVC (default)
1 = HAVC + RF same as video
2 = HAVC + RF different from video
3 = external RF same as video
4 = external RF different from videoQuote:The main idea is to use HAVC_deolddify as a source of "exemplar" frames for "Deep-Exemplar-based Video Colorization" but at the same time to keep the use of HAVC_deolddify in "standalone" mode as in the version 3.5.x."exemplar" frames <= are these what you call RF ?
From the looks of it, I won't add 'DeepExMethod' and 'ScFrameDir' to Hybrid.
Don't see how to properly add these to Hybrid in a meaningful way while making sure they are used as intended.
=> I'll add "EnableDeepEx, DeepExPreset, ScThreshold, ScMinFreq" to HAVC_main.
These setting will not influence the 'custom'-mode at all.
I'll remove: dotweak, dotweak_p, degrain_strength from 'custom'-mode.
Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 26.05.2024
RF is the acronym of "reference frames" and yes it is equivalent to "exemplar frames".
I used this term because it is the term used in "Deep-Exemplar-based Video Colorization"
You should rename the checkbox "DeOldify" at the left of "Coloring" panel in "HAVC" for the reasons already explained (the frame with the DeOldify parameter don't have to be renamed, it's Ok).
I suggest to move the "GPU ID" below Merge weight, so you have more space for DeepEx.
Having more space for DeepEx, you can create a dedicated "DeepEx" frame like "DeOldify" and "DDColor"
In this new frame you can add all the DeepEx parameters, including: method, render_speed, render_vivid, enable_resize, sc_framedir, only_ref_frames, dark, dark_p, smooth, smooth_p, colormap.
As you can note I added in DeepEx some of stabilization parameters, the reason is that when are used inside DeepEx these filters will be applied only on the reference frames and so are much faster.
If it is a problem the proper management of sc_framedir you can limit its usage by forcing the following convention:
if only_ref_frames is true
sc_framedir = clip dir + "\RF_out"
if only_ref_frames is false and method > 0
sc_framedir = clip dir + "\RF_in"
Once you have created this dedicated settings frame for "DeepEx" you can add also 'DeepExMethod', 'ScFrameDir' in the Preset section.
I know that this will complicate the GUI, take all the time you need, there's no rush.
Thanks,
Dan
RE: Deoldify Vapoursynth filter - Selur - 26.05.2024
Atm. one can either use:
1. a preset with HAVC_main which wraps: HAVC_ddeoldify + HAVC_deepex (optional) + HAVC_stabilizer
or
2. (custom) HAVC_ddeoldify + HAVC_stabilizer
Yes, I could adjust Hybrid to convert 'custom' to HAVC_ddeoldify + HAVC_deepex (optional) + HAVC_stabilizer, but I think this is overkill.
Like I mentioned time and time again, the is all getting too complex, too many parameters,...
Especially since I have to expect there to come an additional xy new parameters with the next release of HAVC/deloldify.
Since this is really bothering me, I thought about removing all the custom stuff and just limit Hybrid to support just the HAVC preset based part.
Problem is, that also annoys the hell out of me (only partially supporting havc/deoldify), I now decided to drop the support for it as a whole.
=> Next Hybrid version will not support HAVC/deoldify any more.
If folks want to use it, they would need to either write their own custom additions to Hybrid or write their own code in a custom section.
Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 26.05.2024
I really don't understand your answer.
I don't have intention to add any more parameters.
I can understand that modify the GUI to manage new parameters can be annoying.
But I don't see the reason to throw the baby out with the bathwater.
You have already developed the working code and the filter is working, probably is the most stable coloring filter available on public domain.
You can keep the current version with the limited support to DeepEx, and if one want to use more more complex settings can use the custom section.
Dan
RE: Deoldify Vapoursynth filter - zspeciman - 26.05.2024
Please continue the support for this filter. The amount of thought and creativity that went into this project and the results are astonishing.
There are many of us who are always excited to see the next upgrade to this wonderful tool.
RE: Deoldify Vapoursynth filter - Selur - 26.05.2024
1. I do not have the time and motivation to check the whole code of everything in __init__.py to figure out what inner workings and documentation changed.
2. The amount of parameters is too much, yet in each change it increases.
=> Sorry, it does not seem to be worth the effort for 2 users (Dan64, zspeciman).
RE: Deoldify Vapoursynth filter - Dan64 - 26.05.2024
(26.05.2024, 16:52)Selur Wrote: => Sorry, it does not seem to be worth the effort for 2 users (Dan64, zspeciman).
This post has about 13.532 visualizations so it seems that there are more people interested in this filter.