Deoldify Vapoursynth filter

[Selur]

Updated the deoldify test version download.
DeepExMethod should be included to the HAVC_main-call now. (also fixed the typo)

Cu Selur

[Dan64]

Now the "preset" mode is Ok.

In the "custom" mode, when DeepEx is unchecked the behavior should the same of version 3.5.x

But with these settings


 
Is generated the following code

# adding colors using DeOldify
from vsdeoldify import HAVC_ddeoldify
clip = HAVC_ddeoldify(clip=clip, deoldify_p=[0, 24, 1, 0], ddcolor_p=[1, 24, 1, 0, True], ddtweak=True, ddtweak_p=[0.0, 1.0, 2.5, True, 0.3, 0.6, 0.7, 0.5, "300:360|0.5,0.1"], ScFrameDir="D:/PProjects/HybridAVC/Tests/La Cosa (1951)/frames_color")

Where to HAVC_ddeoldify is added the parameter ScFrameDir that for this function is not available, it is available for HAVC_main() and for HAVC_deepex but with the name "sc_framedir" .

Dan

[Selur]

Updated download, should be fixed now.

[Dan64]

Now is Ok.

As last change, in the "custom" mode, should be allowed to select the "Ref Frame Dir" if in DeepEx is selected the "Method" 1 or 2.

Thanks,
Dan

[Selur]

(updated download)
Shouldn't ScFrameDir in HAVC_main then also be used with method1&2? (it's only set for method 3&4)
btw. the comment text of HAVC_deepex lists sc_framedir 2 times.

Cu Selur

[Dan64]

I download the dev version, but the directory box is still disabled (see picture)
 
In the "preset" mode the directory selection must be enabled if DeepExMethod != 0.

Thanks for reporting the duplication of sc_framedir.

Dan

P.S.
I will not release the version 4.0 this week-end. I still need to review the README to explain the usage of DeepEx.

[Selur]

Okay, something must have gone wrong with the upload, here:

it works. I reuploaded the test version.

Cu Selur

[Dan64]

Now, in "custom" mode, the directory is passed as parameter to HAVC_deepex() , but the parameter "method" is not passed to HAVC_deepex()

Dan

[Selur]

Should be fixed now (updated download)

Cu Selur

Ps.: seems like HolyWu is updating vsFeMaSR (https://github.com/HolyWu/vs-femasr/comm...42af76c1e5)

[Dan64]

Leveraging on the experience in developing vs-deoldify I'm planning to develop a new filter: HybridAVC = Hybrid Automatic Video Colorizer

Where I will integrate: DeOldify, DDColor and Deep-Exemplar based Video Colorization

Deep-Exemplar is able to provide a temporal video stabilization and need to be rewrote in some part to get something useful, is similar to Bistnet, but is simpler and faster.

Since I discovered some coloring problems, I decided to test Bistnet that should be an improved version of "Deep-Exemplar".
Unfortunately I get the same "bad" results. Here an example:



As you can see the colored frame 0000 is very near to the reference frame (0000_ref) in the next frames the colors are propagated very well. The problem happen when in the sequence are introduced new elements not available in the reference frame. In this case are the hands of the woman (see frame 20). In the paper there is written that in this case it will be used the color of the new "feature" (in this case the hands) recovered from the trained network. But as it is possible to see this not the case since the color assigned to the hands is the same as the sweater in the background.    

In this case it is possible to improve the color by providing a reference frame with hands colored correctly. DDeoldiy has not this kind of problems:




So in using HAVC one need to decided if want to use DeepEX to get better stable colors (including green hands) or DDeoldify (Stable) with less stability but with the "features" colored appropriately.

Dan

---

Should be fixed now (updated download)

Cu Selur

Ps.: seems like HolyWu is updating vsFeMaSR (https://github.com/HolyWu/vs-femasr/comm...42af76c1e5)

Now the method is propagated correctly.
In "custom" mode if is selected the DeepEx method 3 or 4, the function HAVC_ddeoldify() don't have to be called, since the output clp_ref is not used. The generated code is correct but is performing a non necessary call to HAVC_deepex().

Dan,

P.S.
Good to know that  HolyWu is updating vsFeMaSR, my problems on mmcv are probably due to the torch version used by Hybrid (2.4) it is possible to downgrade the torch version used by Hybrid to 2.3 ?