Deoldify Vapoursynth filter

[Akila]

I Only have the Stock Hybrid Installed (https://www.selur.de/sites/default/files..._SETUP.exe).

[Selur]

Okay, ignoring most of my last post.... the following might not help.
If you install a system-wide python, that might mess with the portable Vapoursynth, since autoloading can't be disabled in Vapoursynth.
Normally, installing a Hybrid (not installing it over an already existing Hybrid installation) and extracting the torch-addon files into the Hybrid/64bit folder works.
If it does not, usually something is amiss. (might be drivers, or something else on the system)
The first thing to figure out what might be wrong is to look at error messages, which the Vapoursynth Preview usually shows.
If that doesn't help proper details, like mentioned in the sticky, might provide additional hints.

Cu Selur

[Akila]

I uninstalled Python and all what I installed flowed by (https://github.com/dan64/vs-deoldify/) as it was in the thread so I believed I was supposed to follow the instructions.
I uninstalled Hybrid, reinstalled Vanilla Hybrid, extracted "VapoursynthR65_torch_2024.05.11.7z" and I think it is working now.
But only if I select "Custom" -> DeOldify Only". the default presets like "Fast", etc. that are "Simple" (Color Method) I get an empty video file of 500kb, blank picture.

Preview gives me no error (using Simple /or DDcolor): just blank screen / picture

[Dan64]

I guess that you have a hold GPU not supporting fp16 tensor. 
Please uncheck the fp16 checkbox and DDColor should work.

Dan

[Akila]

I guess that you have a hold GPU not supporting fp16 tensor. 
Please uncheck the fp16 checkbox and DDColor should work.

Dan

Yeah, I got GTX 1080 Ti. I turned it off (FP16) and it works in "Custom" preset, but for some reason if I select a preset like "Fast", "Medium" even if FP16 is untick (off), it fails with blank video.
I think when using a preset like "Fast", "Medium", etc. the uncheck of FP16 doesn't actually populate to the script. it only works in Custom preset. Maybe it's a bug

Not Working:





Working:

[Dan64]

In effect the flag "enable_fp16" is not passed to the function ddeoldify_main().

This issue must be fixed by Selur.

Dan

[Selur]

copy&paste error:

this->addOptionIfNotDefault("vsDeoldifyDegrainStrength", "degrain_strength", options);
    this->addOptionIfNotDefault("vsDeOldifyDDFP16", "degrain_strength", options);

should be:

this->addOptionIfNotDefault("vsDeoldifyDegrainStrength", "degrain_strength", options);
    this->addOptionIfNotDefault("vsDeOldifyDDFP16", "enable_fp16", options);

will fix after work.

[Dan64]

Hello Selur

    As promised, I included in vs-deoldify the model: Deep-Exemplar based Video Colorization.
    This model is the father of BiSTNet and provides similar results.
     I rewrote the rendering code and I was able to speed up the rendering from 2fps up to 17fps (depending on the source and settings).
     So this version is fast enough to be used in practice.

    I will write a separate post to describe how to use this model, but now let's starting the practical issues.

    INSTALLATION

     to be able to use this model is necessary to install the following packages.
    

.\python -m pip install scikit-image==0.22.0
.\python -m pip install numba==0.59.1
.\python -m pip install opencv-contrib-python==4.9.0.80

    as usual, is necessary to update the vsdeoldify installation using the attached archive: vsdeoldify-4.0.0_RC1.zip

   To be able to use the new model is necessary to download the file colorization_checkpoint.zip available in the release 1.0: https://github.com/zhangmozhe/Deep-Exemp...s/tag/v1.0 

    the archive have to be unziped in: Hybrid\64bit\Vapoursynth\Lib\site-packages\vsdeoldify\deepex
  
    Finally is necessary to update DDColor with the attached wheel: vsddcolor-1.0.1-py3-none-any.whl.zip

    In this version I added an additional parameter: "scenechange" if this parameter is set will be colored only the frames tagged as scene change frame.

    To test if the installation works it is possible to use the following script:

   

# changing range from limited to full range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=0)
# adjusting color space from YUV420P16 to RGB24
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="709", range_s="full")
# adding colors using HybridAVC
from vsdeoldify import HAVC_main
clip = HAVC_main(clip=clip, EnableDeepEx=True, ScThreshold=0.1)
# changing range from full to limited range for vsDeOldify
clip = core.resize.Bicubic(clip, range_in_s="full", range_s="limited")

   I decided to prefix all the filter functions with the word "HAVC" (Hybrid Automatic Video Colorizer)

   When I started to develop this filter was included only DeOldify, but now there is a lot of additional material: 2 coloring models and pre/post filters.

   To me this name is more appropriate because is including different models, so it can be considered an hybrid model. 
   Moreover this filter is built on the shoulders of "Hybrid", without your application it would be very difficult to use this filter.

   Please let me know if it is working on your side.

Thanks,
Dan

[Bartoloni]

 I was able to speed up the rendering from 2fps up to 17fps (depending on the source and settings).  

wow! very very thank you Dan!

[Selur]

side note:

.\python -m pip install scikit-image==0.22.0
.\python -m pip install numba==0.59.1
.\python -m pip install opencv-contrib-python==4.9.0.80

using strict version dependencies are a bad idea, hope that will not cause problems with other stuff (now and in the future)