+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: Deoldify Vapoursynth filter (/thread-3595.html)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
RE: Deoldify Vapoursynth filter - Dan64 - 03.05.2024
Hello Selur,
I do agree with you that moving inside vsdeoldify the grain/noise removal it will be possible to use a more "bold" approach.
I opted for writing the following degrain function:
def vs_degrain(clip: vs.VideoNode = None, strength: int = 0) -> vs.VideoNode:
if strength == 0:
return clip
else:
import havsfunc
match strength:
case 1:
dstr = 1.0
case 2:
dstr = 1.5
case 3:
dstr = 2.5
case 4:
dstr = 3.5
case _:
raise vs.Error("ddeoldify: not supported strength value: " + mode)
clip = clip.resize.Bicubic(format=vs.YUV444PS, matrix_s="709", range_s="full")
clip = havsfunc.KNLMeansCL(clip=clip, d=1, a=2, s=4, h=dstr, device_type="gpu", device_id=0)
clip = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="full", dither_type="error_diffusion")
return clipI decided to use KNLMeansCL because it is fast and can remove both grain and noise. I simplified the input and to the functions ddeoldify_main() and ddeoldify() I added only 1 parameter, that control the strenght of grain/noise removal, called "degrain_strength" (if = 0, is disabled).
I attached the new RC7.
Thanks,
Dan
RE: Deoldify Vapoursynth filter - Selur - 03.05.2024
Will look at it tomorrow.
RE: Deoldify Vapoursynth filter - Selur - 04.05.2024
Okay, took a look.
Unless I overlook something, this is different from what I suggested. (filtering for deoldify, but keeping the luma details)
=> not implementing this
This is the same as applying the denoising before deoldify which Hybrid already can do, but in Hybrid you can use any denoiser not just KNLMeansCL.
This does make sense when deoldify is used outside of Hybrid, but in Hybrid, adding this seems like a step back.
Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 04.05.2024
The implemented denoise/degrain will preserve the luma details.
The main code is the following:
def ddeoldify(
clip: vs.VideoNode, method: int = 2, mweight: float = 0.5, deoldify_p: list = [0, 24, 1.0, 0.0], ddcolor_p: list = [1, 24, 1.0, 0.0, True], dotweak: bool = False, dotweak_p: list = [0.0, 1.0, 1.0, False, 0.2, 0.5, 1.5, 0.5], ddtweak: bool = False, ddtweak_p: list = [0.0, 1.0, 2.5, True, 0.3, 0.6, 1.5, 0.5, "300:360|0.8,0.1"], degrain_strength: int = 0, cmc_tresh: float = 0.2, lmm_p: list = [0.2, 0.8, 1.0], alm_p: list = [0.8, 1.0, 0.15], cmb_sw: bool = False, device_index: int = 0, torch_dir: str = model_dir) -> vs.VideoNode:
...
...
chroma_resize = True # always enabled
...
...
if chroma_resize:
frame_size = min(max(ddcolor_rf, deoldify_rf) * 16, clip.width) # frame size calculation for inference()
clip_orig = clip;
clip = clip.resize.Spline64(width=frame_size, height=frame_size)
try:
d_clip = vs_degrain(clip, degrain_strength)
except:
vs.core.log_message(2, "Degrain filter: havsfunc.KNLMeansCL is not installed/loaded properly")
d_clip = clip
clip = d_clip
...
...
if chroma_resize:
return _clip_chroma_resize(clip_orig, clip_colored)
else:
return clip_coloredSince it was decided to remove the flag "chroma_resize", now this flag is always enabled.
This implies that no matter all the changes that are made to the luma by the filters, at the end the original luma is always restored.
Dan
P.S.
I discovered a bug in managing the exception on vs_degrain, I attached the correct version.
RE: Deoldify Vapoursynth filter - Selur - 04.05.2024
Ahh,..
chroma_resize_enabled is always true, which calls _clip_chroma_resize, which calls vs_recover_clip_luma
which is:
def vs_recover_clip_luma(orig: vs.VideoNode = None, clip: vs.VideoNode = None) -> vs.VideoNode:
def copy_luma_frame(n, f):
img_orig = frame_to_image(f[0])
img_clip = frame_to_image(f[1])
img_m = chroma_post_process(img_clip, img_orig)
return image_to_frame(img_m, f[0].copy())
clip = clip.std.ModifyFrame(clips=[orig, clip], selector=copy_luma_frame)
return clip
def chroma_post_process(img_m: Image, orig: Image) -> Image:
img_np = np.asarray(img_m)
orig_np = np.asarray(orig)
img_yuv = cv2.cvtColor(img_np, cv2.COLOR_RGB2YUV)
# copy the chroma parametrs "U", "V", of "img_m" in "orig"
orig_yuv = cv2.cvtColor(orig_np, cv2.COLOR_RGB2YUV)
orig_copy = np.copy(orig_yuv)
orig_copy[:, :, 1:3] = img_yuv[:, :, 1:3]
img_np_new = cv2.cvtColor(orig_copy, cv2.COLOR_YUV2RGB)
return Image.fromarray(img_np_new)
Cu Selur
RE: Deoldify Vapoursynth filter - Selur - 04.05.2024
Why are you using havsfunc?
clip = havsfunc.KNLMeansCL(clip=clip, d=1, a=2, s=4, h=dstr, device_type="gpu", device_id=0)clip = core.knlm.KNLMeansCL(clip=clip, d=1, a=2, s=4, h=dstr, channels='Y', device_type="gpu", device_id=0)https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/havsfunc.py#L5948
is only used to filter 'Y' and 'UV' separately to speed it up, but in your case there is no need to filter 'UV' at all.
Cu Selur
Ps.: uploaded a test version which does include KNLMeans, but not havsfunc.
RE: Deoldify Vapoursynth filter - Dan64 - 04.05.2024
Thanks for the advice, I will update the code.
Dan
RE: Deoldify Vapoursynth filter - Selur - 04.05.2024
No, problem will release a new Hybrid tomorrow then, assuming you release a new DeOldify.
Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 04.05.2024
Ok, I will also release the new version tomorrow.
Thanks,
Dan
(04.05.2024, 12:55)Selur Wrote: Ps.: uploaded a test version which does include KNLMeans, but not havsfunc.
I rewrote the "degrain" function as following
def vs_degrain(clip: vs.VideoNode = None, strength: int = 0, device_id: int = 0) -> vs.VideoNode:
if strength == 0:
return clip
match strength:
case 1:
dstr = 1.5
dtmp = 1
case 2:
dstr = 2.5
dtmp = 1
case 3:
dstr = 3.5
dtmp = 1
case 4:
dstr = 5.5
dtmp = 2
case 5:
dstr = 8.5
dtemp = 2
case _:
raise vs.Error("ddeoldify: not supported strength value: " + strength)
clip = clip.resize.Bicubic(format=vs.YUV444PS, matrix_s="709", range_s="full")
clip = vs.core.knlm.KNLMeansCL(clip=clip, d=dtemp, a=2, s=4, h=dstr, channels='Y', device_type="gpu", device_id=device_id)
clip = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="full", dither_type="error_diffusion")
return clipI prefer that in Hybrid the Degrain will display only integer values from 1 to 5.
So that in the future I can decide to implement a different mapping and/or denoise/degrain function, without the need to change the input.
If you prefer you can call this parameter also "DegrainFactor".
Thanks
RE: Deoldify Vapoursynth filter - Selur - 04.05.2024
Uploaded a new test version.
Cu Selur