[BUG] Deep Remaster and Deep Reference modes both crash; ColorMnet reference mode is OK

[Ricks Film Restoration]

Thank you very much for your detailed reply.
Just to say that I tried to add Deep Exemplar to ComfyUI, but in the recent versions they have added some kind of security protections that prevent this.
I tried for days to get it going.

Anyhow, ComfyUI is very complicated, extremely user unfriendly stuff if I may say so. I hope Dan will fix his Vapoursyth colorizer filter. I find it strange that both my PC's have the same crash problem whereas ColorMNet is bug free, but doesn't do a very good job as you saw in my examples. Only the one frame that is used as reference is good in the output video file; the subsequent frames all are bland and have lost a lot of the colors. And yes, I did experiment with all the parameters.

Final comment, if you are in for further improvement: some of the parameters (especially inside the "Exemplar Models" frame) do only appy to some of the three models (ColorMNet, Deep Expemplar or Deep Reference), but remain visible even when not applicable. It would be better (not "grey/inhibit them) but to hide them when not valid: 

I have no idea when/why the "Reference Frames Only" check box has any function, what the four "SC" parameters do during ColorMNet, the "Weight"parameter, nor what the DDColor checkbox is supposed to do (in combination with HAVC).

Also the difference between the "HAVC+RF same as video" and " RF same as video" is unclear to me. If there is no difference then better to remove one of them.
The "Help" shows nothing when hovering over the 8 choices in the "Methods" pull down list. This, plus the "Vivid" checkbox and the "Preset" (fast, medium, slow) setting don't show noticeable differences in the Output video. I don't think I am doing something wrong.

There are more things that I could report to you, but am not sure whether you are open to such feedback, considering that you stated to be "happy with your UI as it is"...

[Ricks Film Restoration]

Hi Dan,

Thank you very much for your explanation. As you probably saw in my other post I am not very satisfied with the ColorMNet results (but at least it doesn't crash!). For that reason I want to use Deep Exemplar mode, but it crashes on both my PC's as described earlier. Deep Remaster also crashes.
Both PC's are made by HP with Nvidia RTX 3090 and RTX 4070 GPU's respectively.

I can't believe I am the only one with these crash problems, unless no-one is active with reference frame colorizations. Please check whether everything is still OK and give me some advise what to do.

I used Deep Exemplar a number of times with success via Github. The main problem is that the resolution is then only 768 x 432 pixels, completely spoiling the B&W AI enhancements to 1080p that I do with Topaz Starlight/Astra. Deep Exemplar retains detailed colorization info like uniform medals whereas in ColorMNet it becomes already washed out in the next frame.

---

Hi,

Just to comment on Topaz Starlight / Astra: the past year these new models give superior results compared to earlier colorization models. Faces are no longer deformed. Topaz now uses diffusion AL technology. However, Topaz has become very expensive and since October are only offering their software on a monthly/annual subscription basis.

DeOldify has always been my favorite. I used it for 5 years via Github without major problems. Sadly it is now broken on Github and its maker, Jason Antic, has stopped supporting it. 
There is still a remnant of his work on MyHeritage's photo colorize feature, but has nasty red blotches (in stead of the annoying purple haze that has always been a problem of DeOldify.

Fortunately DeOldify has been revived (by Dan64?) in the HAVC Vapoursynth Colorization filter. However, I prefer the "Video+ModelScope" mode over DeOldify: nicer colours.

Watch my Rick88888888 Youtube channel (300+ historic videos) to see what progress I have made over the past decade.

[Dan64]

I'm sorry but without debug info it is difficult to understand where is the problem.

But you can do the following test:

• After having set the HAVC settings, instead of encoding the movie, push the button preview (1) as described in my HAVC guide at page 55-56.
  

• If you are able to see the colored frames, this imply that there some other problem and in this case it is mandatory to provide the debug file (enable the debug file by checking Config->Internals->Create debug file).
  
  
• If you are not able so see the colored frames, (black screen), should be displayed a lot of messages, please copy and post them. They will help me to understand the problem.
  

Dan

[Dan64]

Watch my Rick88888888 Youtube channel (300+ historic videos) to see what progress I have made over the past decade.

I watched your last video Fascinating footage of the London Underground during WWII, at Christmas in 1940

The clip seems to be colored using some AI model, but what impressed me is the color consistency. I guess that you have generated some reference frames and then propagated them using a model like Deep-Remaster. Could you provide more details on the generation of this movie ?

Thanks,
Dan

[Selur]

@Dan64: Aren't those just animated images? (probably 'AnimateDiff' with ComfyUI)

Cu Selur

[Ricks Film Restoration]

First of all I'd like to say that I find it a pity that the thread about "The Night To Remember" has now been closed. I watched your version: great work!
I have seen that happen in other forums too when discussions that start to be interesting get cut off....
By the way, I didn't bring up ComfyAI: I don't like it at all: far too complicated. Thanks for the tip to use it in portable mode. My 'money' is now on Hybrid + HAVC.
Two years ago I did "The Dam Busters" (1957) movie entirely with DeOldify (and upscaled to 1080p), but I didn't dare to put it on my Archive.org channel because of Copyright.
I can WeTransfer it to you, if you like.

To answer your question: the film about the London Underground was made by colorizing original B&W photos with Google Gemini (which uses Nano Banana).
It is still riddled with bugs, fantasizes and hallucinates far too often despite telling it not to in the prompt. Colorization is also inconsistent: some results are great but half the time it generates almost B&W results or hardly any colors. On average every fourth image it generates a completely random crap image, nothing to do with the prompt. Gemini admits it has flaws, due to over-complicating the algorithm that looks too far back in the Chat history and thus gets stuck in loops. Google is very lacks in solving these bugs.

Anyhow, the Gemini Nano Banana results are first passed on to Paintshop Pro (V7) which I use to manually get rid of the Gemini diamond shaped icon (bottom right). Then it is fed to various image-to-video AI's to create a 5 or 10 seconds video. For that I use Pixverse, Kling and (previously) Hailuoai (Minimax).
Color consistency is very good with these AI's. Pixverse has 60 daily credits, enough to create 2 videos of 5 seconds a day in 540p for free.

Gemini is great to colorize reference frames to be used with HAVC+Hybrid.
I then use TOPAZ video AI or AIarti to upscale the result from 540p to 1080p HD.

I know this has nothing to do with Hybrid, but I thought I should answer your question.
FYI: I don't like it that about 10% of my Youtube audience calls videos like the one about London: "AI slop" which it isn't. Viewers have more and more difficulty to distinguish between text-to-video AI generated "slop" and image-to-video AI based on actual photos.
My next video will be about Spitfire dogfights during the London Blitz, by using original B&W historic footage colorized with Hybrid + HAVC. So Selur will be pleased with that.

You can contact me via ricksfilmrestoration@gmail.com about anything unrelated to Hybrid in order not to upset Selur. Could/should be interesting for both of us....

[Selur]

I have seen that happen in other forums too when discussions that start to be interesting get cut off....

If you invade a thread of another user and start to go off-topic instead of creating a separate thread in a forum where such a discussion belongs, this is what happens.

about anything unrelated to Hybrid in order not to upset Selur. Could/should be interesting for both of us....

there is a sub-forum for such stuff and the ' Hybrid - Support › Problems & Questions' is not it.

Cu Selur