+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: Deoldify Vapoursynth filter (/thread-3595.html)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
RE: Deoldify Vapoursynth filter - Dan64 - 24.03.2024
(24.03.2024, 12:22)Selur Wrote: Ah, post wasn't finished. (Images were just shown without explaination as attachment)
I substitute the last line with
clip = clipMask.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="limited", dither_type="error_diffusion")And I confirm you that the mask is always "white".
I'm planing to write my own motion-mask filter...
Dan
RE: Deoldify Vapoursynth filter - Selur - 24.03.2024
Ah okay.
Something like:
deviation= 0.05
steps: int = 1
tht = 10
max_steps = max(min(steps, 5), 1)
clip_limited = vs_clip_chroma_stabilizer(clip, deviation=deviation)
for i in range(1, max_steps):
clip_limited=vs_clip_chroma_stabilizer(clip_limited, deviation=deviation)
# calculate motion mask
org = clip
clipMask = clip
clipMask = clipMask.resize.Bicubic(format=vs.GRAY8, matrix_s="470bg", range_s="limited")
clipMask = vs.core.motionmask.MotionMask(clip=clipMask, th1=tht, th2=tht, tht=tht) # pixels with abs(diff) < tht will be black (static parts)
clipMask = vs.core.std.InvertMask(clip=clipMask) # invert so that static parts are white (weight=1)
# merge in YUV color space
clipMask = clipMask.resize.Bicubic(format=vs.YUV444PS, matrix_s="470bg", range_s="limited")
clip_limited = clip_limited.resize.Bicubic(format=vs.YUV444PS, matrix_s="470bg", range_s="limited")
clip = clip.resize.Bicubic(format=vs.YUV444PS, matrix_s="470bg", range_s="limited")
clip = vs.core.std.MaskedMerge(clipa=clip, clipb=clip_limited, mask=clipMask) # MotionMask
# restore RBG24 color space
clip = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="limited", dither_type="error_diffusion")
clipMask = clipMask.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="limited", dither_type="error_diffusion")
org = org.resize.Bicubic(format=vs.RGB24, matrix_in_s="709", range_s="limited", dither_type="error_diffusion")
clip = core.std.StackVertical([
core.std.StackHorizontal([org.text.Text("Original"), clipMask.text.Text("Mask")]),
core.std.StackHorizontal([clip.text.Text("Filtered"), clip.text.Text("Filtered")])
])
And I confirm you that the mask is always "white".
I'm planing to write my own motion-mask filter...RE: Deoldify Vapoursynth filter - Dan64 - 24.03.2024
I confirm you, that this filter is flawd. As been written (or tested) to work in an narrow range of "tht=10"
Here the mask with tht=10:
Here the inverted mask:
If I increase "tht" the inverted mask becomes "white", this is not what I want. The expectation is that by increasing "tht" the moving parts will be considered "static", but the filter is unable to get this result.
Dan
RE: Deoldify Vapoursynth filter - Selur - 24.03.2024
Hmm,... I agree higher values should detect less changes.
Cu Selur
RE: Deoldify Vapoursynth filter - Selur - 24.03.2024
I updated the Hybri deoldify test version, which should now work with v2.0.1
(hope I adjusted to all the changes
)Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 24.03.2024
The new dev-version is working!
I noted that the new temporal filter "color_limiter" is penalized in Vapoursynth encoding pipeline.
For example the encoding speed on my PC with SimpleMerge
clip = ddeoldify(clip=clip, model=0, sat=[1.00,1.00], hue=[0.00,0.00], chroma_resize=True, dd_method=2)is 5.28fps
The encoding speed with all post process filters enabled, excluding only "color_limiter"
clip = ddeoldify(clip=clip, model=0, sat=[1.00,1.00], hue=[0.00,0.00], chroma_resize=True, dd_method=2, dark_darkness=[True,0.1,0.2,0.6,0.65], color_stabilizer=[True,False,True,5,"arithmetic",True])is 5.20fps, only 2% slower.
But if I add the temporal filter "color_limiter"
clip = ddeoldify(clip=clip, model=0, sat=[1.00,1.00], hue=[0.00,0.00], chroma_resize=True, dd_method=2, dark_darkness=[True,0.1,0.2,0.6,0.65], color_stabilizer=[True,False,True,5,"arithmetic",True], color_limiter=[True,0.02])The speed decrease to 2.58fps, 52% slower
But if I apply the filter after "ddeoldify"
clip = ddeoldify(clip=clip, model=0, sat=[1.00,1.00], hue=[0.00,0.00], chroma_resize=True, dd_method=2, dark_darkness=[True,0.1,0.2,0.6,0.65], color_stabilizer=[True,False,True,5,"arithmetic",True])
clip = dd_video_stabilizer(clip=clip, chroma_resize=[True,24], color_limiter=[True, 0.02])The encoding speed is 5.04fps, only 5% slower.
It is not clear the reason of such difference in speed, but being a "temporal" filter, it is penalized if applied with other filters in the same pipeline.
I don't know if you are able to observe the same decrease in speed, but if so, It could be worth making available in Hybrid, as a post-post process filter also "dd_video_stabilizer".
Thanks,
Dan
RE: Deoldify Vapoursynth filter - Selur - 24.03.2024
Yes, I get the similar speed inpacts.
I agree that calling:
1. ddeoldify
and then
2. dd_video_stabilizer does make sense
does seem like better approach.
I'm really reluctant to add dd_video_stabilizer since, it's the whole thing gets more and more unusable.
This would add another 14+ ui elements,...
Would it make sense to remove color_stabilizer, color_limiter from ddeoldify
(this would remove the possiblity to use color_stabilizer for ddeoldiy and ddcolor separately) and then call
1. ddeoldify(..)
and then
2. dd_video_stabilizer(..) in case color_stabilizer, color_limiter or color_smoothing is enabled
?
Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 24.03.2024
I understand your point of view.
The more conservative approach is to don't touch the GUI, but in the case is enabled "color_limiter" it can be run separately using dd_video_stabilizer(..) after ddeoldify(..).
For the moment is better enable this separate process only for "color_limiter".
I'm still working on a Vapoursynth version "color_stabilizer" which is not introducing gray frames.
In the case I will be able to find a working solution, it would be better run this version of filter separately like "color_limiter" because is a temporal filter.
Thanks,
Dan
RE: Deoldify Vapoursynth filter - Selur - 24.03.2024
Uploaded a new dev version (same link), let me know what you think of this one.
(loads ddeolify and then dd_video_stabilizer)
Cu Selur
RE: Deoldify Vapoursynth filter - Dan64 - 24.03.2024
The GUI It's more rational, I like it.
Also the speed is good.
Thanks,
Dan
In the next days I will release a new version (only to you), where I will clean up the ddeoldify filter, by removing the post process filters and probably will rename "dd_video_stabilizer" in "ddeoldify_stabilizer".
Dan