Using Stable Diffision models for Colorization

[Dan64]

Hi safshe,

  The GUI has 4 tasks (the last is optional and suitable only if is used in input a clip already colorized).

   

   You can skip the first task and generate the reference frames using other external tools. It is only necessary to assign them the right name (ref_nnnnnn.jpg) and put them in the folder called "ref_tht10". Then you can start the colorization using Qwen-IE.

   

    Having split the pipeline in independent tasks will provide more flexibility. After the colorization you can check the colored frames and eventually adjust them.
    Finally you can run the full clip colorization using CMNET2 by running the task #3.

Dan

P.S.

You can add other extraction scripts in the folder "GUI\scripts", to be visible they need to be named *extract*.vpy.  If you want to use other filters for scene-detection, if they are available as VS filter you can add them in your (.venv) and create a new script to use them, look at the script extract_refs_edge.vpy as an example.      

You can add also other colorization scripts it is just enough to rename them *encode*.vpy. If you want to use the colorization functions available in HAVC it is just enough to install the filter in your (.venv) environment (remember to add all the necessary model weights). Look at the script encode_cmnet2.vpy as an example.