Selur's Little Message Board
Deoldify Vapoursynth filter - Printable Version

+- Selur's Little Message Board (https://forum.selur.net)
+-- Forum: Talk, Talk, Talk (https://forum.selur.net/forum-5.html)
+--- Forum: Small Talk (https://forum.selur.net/forum-7.html)
+--- Thread: Deoldify Vapoursynth filter (/thread-3595.html)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81

RE: Deoldify Vapoursynth filter - Selur - 02.03.2024

Quote:In Stable Diffusion it is possible to set for every "filter" a weight that they call "visibility".
I understand that implementing this feature in Hybrid for every filter is a mess.
Main problem is, I honestly have no clue how to implement this aside from the way the Merge function in Hybrid already does.
(enable "Filtering->Misc->UI->Show 'Merge'-controls")
For neural networks adding 'weights' is kind of the whole idea,.. Wink

Quote:But at least you can consider the possibility to Merge the 2 filters in some way.
Again, I don't know how to 'merge' stuff in a meaningful/good way. (+ how to add this to the ui without making the ui even more unapproachable to users Wink)
Using http://www.vapoursynth.com/doc/functions/video/merge.html doesn't seem like a really promising way.

Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 02.03.2024

I can try to implement an approach like CodeFormer, where in input is provided a "clipBG".
In this case the "clipBG" will be the clip colored with ddcolor, then I will merge them is some way, maybe there is some optimal way to do this kind of merge, I need to think about it.
But in any case for Hybrid the merge will be delegate to "ddeoldify". This should reduce the complexity to add this feature in Hybrid.

Dan

RE: Deoldify Vapoursynth filter - Selur - 02.03.2024

Let me know if you come up with a good way to merge stuff, even if it's just for ddcolor&deoldify. Smile
[Image: grafik.png]
But I think that both might be better than b&w, there is still much room for improvement so we probably will have to see where the general development goes.

Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 02.03.2024

I will integrate "ddcolor" in "ddeoldify". Cool
I will add more parameters to control the merge.
I will inform you when the new version will be available.

Dan

RE: Deoldify Vapoursynth filter - Selur - 02.03.2024

Sounds interesting Smile

RE: Deoldify Vapoursynth filter - Dan64 - 02.03.2024

I released a new version: https://github.com/dan64/vs-deoldify/releases/tag/v1.1.0

New parameters has been added to control the combination of 2 models.
The parameters added are:
  1. dd_weight: weight assigned to ddcolor, if = 0 ddcolor will be disabled (default = 0) [range: 0-1]
  2. dd_strength: ddcolor input size, if = 0 ddcolor will be disabled (default = 1) [range: 0-4]
  3. dd_model: ddcolor model: 0 = ddcolor_modelscope, 1 = ddcolor_artistic (default = 0)
  4. dd_num_streams: number of CUDA streams to enqueue the kernels (default = 1)

The strategy adopted to combine the 2 models is a simple Merge.
I will evaluate the possibility to improve the combination of the 2 models (it would be an option to de-saturate one of the 2 models)

I hope that you can integrate the new parameters in Hybrid.

Thanks,
Dan

RE: Deoldify Vapoursynth filter - Selur - 02.03.2024

Will look at it tomorrow. Smile

Quote:The strategy adopted to combine the 2 models is a simple Merge.
So not that useful, you might want to look into using color based masking and merging.

Quote: I will evaluate the possibility to improve the combination of the 2 models (it would be an option to de-saturate one of the 2 models)
fingers crossed Smile

Cu Selur

RE: Deoldify Vapoursynth filter - Dan64 - 03.03.2024

I released a new version: https://github.com/dan64/vs-deoldify/releases/tag/v1.1.1

I implemented the following changes:
  •   now is possible adjust the saturation and hue of the 2 models with the parameters sat[], hue[], where the first element of array will be applied to Deoldify and the second to DDColor.
  •   when dd_strenght=0 will be automatically calculated the optimal size (default=0)
  •   the parameter dd_method will allow to select the merge mathod, currently only Simple Merge is implemented.

I hope that you will be able to include in Hybrid all the parameters (that now are increased to 13).

Thanks,
Dan

RE: Deoldify Vapoursynth filter - zspeciman - 03.03.2024

This sounds like an exciting project.  Is it for colorizing photos only or also videos?

RE: Deoldify Vapoursynth filter - Selur - 03.03.2024

@Dan64: I'll look at it later.
@zspeciman: In Hybrid DeOldify (like ddcolor) will be applied to the video, DeOldifys 'Video'-Model was meant for video. Both of them are nowhere near perfect, so on some content it might work good, while on others not so much.
(see the link in #40 and the screenshots throughout this thread)

Cu Selur