Using Stable Diffision models for Colorization

[safshe]

As a natural extension of the "Fix Image" tab, I've added the new "Fix Video" tab.

Suppose you've already colorized a clip with CMNET2 and DitServerRPC. There will be a folder called "ref_qwen" containing the colored frames.
Suppose you've viewed the clip and want to change the color of some frames. Let's say these frames range from #34 to #525. After fixing them with the "Fix Image" tab, you can recolor the video only in the identified frame range (in this case, from #34 to #525). As shown in the image below.


I hope this helps; in some cases, it was necessary for me (which is why I added the implementation).

Since for technical reasons, I still need to re-encode the entire clip, it's mandatory to use as input the clip already colorized and to have NVEnC installed as explained in the README.

Dan

Please add a new feature to the Fix Image pipeline.
Proposed Workflow:

• Step 1: Load a pre-colorized image to act as the primary color base.
  
• Step 2: Select a new reference image that you want to colorize.
  
• Step 3: Perform the colorization on the reference image, but strictly guide the colorization based on the palette and tones of the already loaded colorized frame.
  
  

I have tested a similar workflow in ChatGPT, and the logic works well. I hope we can implement this

[Dan64]

Hi safshe,

   you must try with ComfyUi. 
   If it works with ComfyUi there is some possibility that it could be added in the GUI of DiTServerRPC.

Dan