Deoldify Vapoursynth filter

[zspeciman]

Around 15-20% faster, that's always good news.  Any thoughts about this Google Colorization Transformer?
https://github.com/google-research/googl...er/coltran

There is an article comparing it to Deoldify
https://habr.com/en/companies/ruvds/articles/568426/

[Dan64]

yes I read the article, the conclusion is that Deoldify is better. But the comparison is unfair, ColTran is using very large networks, which is not suitable for practical encoding, because the speed will be around 0.001fps. Despite the larger networks used by ColTran, Deoldify was on average significantly better than ColTran.

Dan

---

    if clip.format.id != vs.RGB24:
        # clip not in RGB24 format, it will be converted
        clip = clip.resize.Bicubic(format=vs.RGB24, range_s="limited")

did you test this with different inputs?
iirc. if you do YUV to RGB conversions (or the other way around), one needs to specify the input matrix.

I found a bug in the function adjust.tweak() due to a typo (I written "clp" instead of "clip").
I released a new version fixing it: https://github.com/dan64/vs-deoldify/rel...tag/v1.1.4

Thanks,
Dan

[Selur]

ARGH,... just spend an hour uploading a new torch addon,..
(and there goes another 1 1/2 hours packaging and updating )
But thanks for the info about the fix.

Cu Selur

[Dan64]

I update the README to include Hybrid support: https://github.com/dan64/vs-deoldify

Thanks again for your help
Dan

[Selur]

No need to thank me, I'm thankful to add a new 'plaything' to Hybrid.
btw. if you are looking for a new project https://github.com/sczhou/ProPainter could need a Vapoursynth wrapper.

Cu Selur

[Dan64]

I'm working on writing a comparison of Deoldify, DDColor, ColTran and DDeoldify. I will post the results on you "Small Talk" section of your forum.  

Dan

P.S.
I will look to ProPainter later.

[zspeciman]

Great work to the both of you.  I've got it working as well.  
I've run a video comparison between regular Deoldify and the Hybrid Deoldify.  For some reason the Hybrid version is a little brighter, not as dark.
Where is the setting to fix that?

[Selur]

Using a tv scale source.
Adding:

clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)

before applying vsdeoldify, doesn't seem to change anything.
But using:

# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

does lighten the image up more.

So it might be a tv vs. pc scale problem.
If you use Tweak on full scale input, make sure to disable 'Coring', since that will limit the input to tv scale.

If you use an image as source: try different luma range setting.

=> would need the source to say for sure where the problem is

Cu Selur

[Dan64]

Great work to the both of you.  I've got it working as well.  
I've run a video comparison between regular Deoldify and the Hybrid Deoldify.  For some reason the Hybrid version is a little brighter, not as dark.
Where is the setting to fix that?

I published in https://github.com/dan64/vs-deoldify
a Model Comparison section, where you can find the Hybrid setup used and the test images.

Dan

---

Around 15-20% faster, that's always good news.  Any thoughts about this Google Colorization Transformer?
https://github.com/google-research/googl...er/coltran

There is an article comparing it to Deoldify
https://habr.com/en/companies/ruvds/articles/568426/

I published in https://github.com/dan64/vs-deoldify
a Model Comparison section, using as reference images the ones published in the Habr article. Instead of using a subjective measure to evaluate the color differences I used a quantitative method using the CIEDE2000 color similarity metric. In the test the ColTran + TensorFlow 2.4.1 was the model that performed worse.

Dan

[zspeciman]

@Dan64, that was a nice test you run.  I was colorizing some photos and videos as well to see the difference.  DDcolor photo images are stunning, but in the videos, in some parts it works very well (more robust color than standalone DeOldify) and in others parts more like the 60s psychedelic colors.  The merge concept is a brilliant idea to combine the stability of DeOldify with the the color pop of DDcolor

I have few questions about some of the settings, as they relate to video
In DDcolor, what is Input size about?  FP16?  Artistic Model vs ModelScope?
I wasn't sure what Streams settings was about either, but when I changed 1 to 4, the video had a several corrupted images, so I stuck to 1


In DeOldify with Simple Merge enabled, in the DDcolor settings on the right, what does that input size about?