Deoldify Vapoursynth filter

[Selur]

In Stable Diffusion it is possible to set for every "filter" a weight that they call "visibility".
I understand that implementing this feature in Hybrid for every filter is a mess.

Main problem is, I honestly have no clue how to implement this aside from the way the Merge function in Hybrid already does.
(enable "Filtering->Misc->UI->Show 'Merge'-controls")
For neural networks adding 'weights' is kind of the whole idea,..

But at least you can consider the possibility to Merge the 2 filters in some way.

Again, I don't know how to 'merge' stuff in a meaningful/good way. (+ how to add this to the ui without making the ui even more unapproachable to users )
Using http://www.vapoursynth.com/doc/functions...merge.html doesn't seem like a really promising way.

Cu Selur

[Dan64]

I can try to implement an approach like CodeFormer, where in input is provided a "clipBG".
In this case the "clipBG" will be the clip colored with ddcolor, then I will merge them is some way, maybe there is some optimal way to do this kind of merge, I need to think about it.
But in any case for Hybrid the merge will be delegate to "ddeoldify". This should reduce the complexity to add this feature in Hybrid.

Dan

[Selur]

Let me know if you come up with a good way to merge stuff, even if it's just for ddcolor&deoldify.

But I think that both might be better than b&w, there is still much room for improvement so we probably will have to see where the general development goes.

Cu Selur

[Dan64]

I will integrate "ddcolor" in "ddeoldify".
I will add more parameters to control the merge.
I will inform you when the new version will be available.

Dan

[Selur]

Sounds interesting

[Dan64]

I released a new version: https://github.com/dan64/vs-deoldify/rel...tag/v1.1.0

New parameters has been added to control the combination of 2 models.
The parameters added are:

1. dd_weight: weight assigned to ddcolor, if = 0 ddcolor will be disabled (default = 0) [range: 0-1]
   
2. dd_strength: ddcolor input size, if = 0 ddcolor will be disabled (default = 1) [range: 0-4]
   
3. dd_model: ddcolor model: 0 = ddcolor_modelscope, 1 = ddcolor_artistic (default = 0)
   
4. dd_num_streams: number of CUDA streams to enqueue the kernels (default = 1)
   

The strategy adopted to combine the 2 models is a simple Merge.
I will evaluate the possibility to improve the combination of the 2 models (it would be an option to de-saturate one of the 2 models)

I hope that you can integrate the new parameters in Hybrid.

Thanks,
Dan

[Selur]

Will look at it tomorrow.

The strategy adopted to combine the 2 models is a simple Merge.

So not that useful, you might want to look into using color based masking and merging.

I will evaluate the possibility to improve the combination of the 2 models (it would be an option to de-saturate one of the 2 models)

fingers crossed

Cu Selur

[Dan64]

I released a new version: https://github.com/dan64/vs-deoldify/rel...tag/v1.1.1

I implemented the following changes:

•   now is possible adjust the saturation and hue of the 2 models with the parameters sat[], hue[], where the first element of array will be applied to Deoldify and the second to DDColor.
  
•   when dd_strenght=0 will be automatically calculated the optimal size (default=0)
  
•   the parameter dd_method will allow to select the merge mathod, currently only Simple Merge is implemented.
  

I hope that you will be able to include in Hybrid all the parameters (that now are increased to 13).

Thanks,
Dan

[zspeciman]

This sounds like an exciting project.  Is it for colorizing photos only or also videos?

[Selur]

@Dan64: I'll look at it later.
@zspeciman: In Hybrid DeOldify (like ddcolor) will be applied to the video, DeOldifys 'Video'-Model was meant for video. Both of them are nowhere near perfect, so on some content it might work good, while on others not so much.
(see the link in #40 and the screenshots throughout this thread)

Cu Selur