Deoldify Vapoursynth filter

[Dan64]

Hello Selur,

  I discovered that weird things happen in vs-deoldify when is used the default VS multi-threading.
  When is enabled "Exemplar Models" it is necessary to disable it with "core.num_threads=1".
  This issue was reported also on EGVSR in your post: VSGAN - VapourSynth GAN Implementation, based on ESRGAN

  I tried to use the statement  "core.num_threads=1" in my functions, but is not working.
  It works only if is inserted at beginning of the script produced by Hybrid as:

clip = core.bs.VideoSource(source="sample.mp4")
core.num_threads=1
frame = clip.get_frame(0)

  It is feasible for you to add this feature for HAVC ?

Thanks,
Dan

[Selur]

Will try to look into it tomorrow.
Should be easy to fix.
(You can set "Filtering->Vapoursynth->Misc->Script->Limit thread count" to 1 as a workaround until I get to it.)

[Selur]

Uploaded a new dev version which enforces single threading when "Exemplar Models" is enabled.

Cu Selur

[Dan64]

I installed the new dev (**experimental**) version and it is working. 
Thanks!

Unfortunately the multi-thread problem can theoretically impact on any filter that is depending on past/future frames because the frame sequence is not guaranteed.

I also observed that the VS function used for scene detection "misc.SCDetect()" has an odd behaviour even in single-thread, for example the number of scene change (SC) detected with threshold=0.02 is lower than the number of SC obtained with threshold=0.05. I rewrote the function using "core.std.PlaneStats()" and the behaviour is correct, with threshold=0.02 I obtain about twice the number of SC detected with threshold=0.05. It is about 5% slower, but it works.

I'm planning to release a new version 4.5.1 with an improved scene detection function. 
This new version will introduce 2 new parameters to better control the scene change behaviour.

Do you have any plan to release the new **experimental** version 2024.XX.XX.1 ?

Dan

[Selur]

I originally planned to do a new release for next weekend, but that won't happen. (The monitor on my main system broke down, and I didn't have much time to code this week.) I'll probably release a new version the weekend after the next.

Scene change with a value of 0.02 or 0.05 is insane, this will cause tons of false positives.
As a side note: Have you ever tried calling retinex on a source before trying scene change detection. Also scene change detection usually should be done with cropped sources.


Cu Selur

[Dan64]

If you can delay the release is better, I'm still working on improving the scene change detection algorithm.

I know that a threshold 0.02/0.05 is insane, but this setting is coupled with other procedures that filter out the frames not necessary.
I will add 2 additional parameters to control the filter process of frames not necessary.
This approach will allow to speed-up the coloring process with HAVC of about 2x.

Unfortunately the single thread approach is slowing down too much the encoding.
It is better to add the single thread as an option in the Coloring Panel (to avoid to set it globally with the Config).

Thanks,
Dan

[Selur]

It is better to add the single thread as an option in the Coloring Panel (to avoid to set it globally with the Config).

It sounded like single threading was necessary for DeOldify with "Exemplar Models" to work properly.
So, is this not the case?

If you can delay the release is better, I'm still working on improving the scene change detection algorithm.

Sure, I can delay the release.

Cu Selur

[Dan64]

It is better to add the single thread as an option in the Coloring Panel (to avoid to set it globally with the Config).

It sounded like single threading was necessary for DeOldify with "Exemplar Models" to work properly.
So, is this not the case?

The single thread approach is necessary to get the Scene Detection algorithm to work perfectly. In the multi thread setting it will generate more frames than necessary, but the coloring process will work in any case. Given the significant slow down of the single thread approach, it is better to add it as an option, so that people not having a powerful GPU can still be able to use the "Exemplar Models" with a decent speed (above 3/4 fps).

Dan

[Selur]

Maybe try something like a preroll like used with https://github.com/WolframRhodium/muvsfunc/issues/48 help?

[zspeciman]

I colorized a short very dark clip after first applying Retinex to it (default settings).  The colors turned out great, but the video is a bit too bright.  Is there a way to reverse the Retinex effect and apply it on the colorized clip?