Deoldify Vapoursynth filter

[Dan64]

Leveraging on the experience in developing vs-deoldify I'm planning to develop a new filter: HybridAVC = Hybrid Automatic Video Colorizer

Where I will integrate: DeOldify, DDColor and Deep-Exemplar based Video Colorization

Deep-Exemplar is able to provide a temporal video stabilization and need to be rewrote in some part to get something useful, is similar to Bistnet, but is simpler and faster.

Since I discovered some coloring problems, I decided to test Bistnet that should be an improved version of "Deep-Exemplar".
Unfortunately I get the same "bad" results. Here an example:



As you can see the colored frame 0000 is very near to the reference frame (0000_ref) in the next frames the colors are propagated very well. The problem happen when in the sequence are introduced new elements not available in the reference frame. In this case are the hands of the woman (see frame 20). In the paper there is written that in this case it will be used the color of the new "feature" (in this case the hands) recovered from the trained network. But as it is possible to see this not the case since the color assigned to the hands is the same as the sweater in the background.    

In this case it is possible to improve the color by providing a reference frame with hands colored correctly. DDeoldiy has not this kind of problems:




So in using HAVC one need to decided if want to use DeepEX to get better stable colors (including green hands) or DDeoldify (Stable) with less stability but with the "features" colored appropriately.

Dan

---

Should be fixed now (updated download)

Cu Selur

Ps.: seems like HolyWu is updating vsFeMaSR (https://github.com/HolyWu/vs-femasr/comm...42af76c1e5)

Now the method is propagated correctly.
In "custom" mode if is selected the DeepEx method 3 or 4, the function HAVC_ddeoldify() don't have to be called, since the output clp_ref is not used. The generated code is correct but is performing a non necessary call to HAVC_deepex().

Dan,

P.S.
Good to know that  HolyWu is updating vsFeMaSR, my problems on mmcv are probably due to the torch version used by Hybrid (2.4) it is possible to downgrade the torch version used by Hybrid to 2.3 ?